Tinderbox shows a 1.2MB increase in the optimized codesize when bug 388403 landed. This is as compared to the total 1.6MB size of the NSS object files on a comparable Linux system before that patch. This increase is a 9% increase in the total codesize of the application. See also bug 388403 comment 17 for some ideas on possibly addressing this regression. Basic breakdown of codesize changes by library: libnss3.so Total: +1038686 (+1046387/-7701) libnssdbm3.so Total: +176985 (+176985/+0) libfreebl3.so Total: +35044 (+35860/-816) libnssckbi.so Total: +20062 (+20844/-782) libssl3.so Total: +1600 (+1646/-46) shlibsign Total: +464 (+1447/-983) libsmime3.so Total: +256 (+1202/-946) libxul.so Total: -368 (+453/-821) libsoftokn3.so Total: -74649 (+37488/-112137)

This can be used to look for things to optimize.

Should we back this out until we resolve? that's a big codesize hit...

Most of the code size increase is due to libPKIX. And some is due to the addition of code to work with the new sharable DBs. But the resolution is certainly not going to involve abandoning libPKIX, nor shared DBs. In the past 6 years, the world of IETF PKI standards has evolved a lot. As Kai noted in bug 388403, certificates are no longer organized in simple independent acyclic trees, but now are networks. In the old world of certs, any cert could have only one issuer, but in the new world, certs may have multiple parents. Walking the graph from a user cert to a "root" CA cert is no longer a simple linear walk, but is now a full tree walk. Further, the standards have evolved to allow the parent certs for a cert to be fetched over the web via URLs in the certs themselves. Consequently, the logic for constructing a "chain" (or "path") of certs to validate, and validating that chain, has gotten orders of magnitude more complex. NSS fell behind the state of IETF RFCs, and now is playing "catch up" with the addition of libPKIX. libPKIX has taken a team of developers 3 years to develop, and presently is over 60K lines of library code, and a similar amount of QA test code (not in libPKIX itself). libPKIX includes new code to fetch parent certs and CRLs via http and/or LDAP. It does full blown policy processing and also handles "bridge CAs". Most of this new stuff is needed for EV. IMO, the present code is good enough for a FF "alpha". We will undoubtedly be working to reduce the size of the code, (without reducing the functionality of the code). It may well be that the size of libPKIX can be reduced by turning lots of macros into functions, as Boris suggested in bug 388403 comment 17. I don't have much of a feel for how much reduction that would actually buy us though.

Oh, and I should mention that Windows already has most (if not all) of this new functionality in Vista (some in XP). This is buying feature parity now.

By the way, I would probably use this as a tracking bug and file bugs blocking this on on particular code work. For what it's worth, to test how correct bug 388403 comment 17 is, I just tried the following experiments (with rebuilds after each one): 1) Fix issue 4 in that comment in just PKIX_CHECK 2) Make PKIX_CHECK a no-op Experiment 1 showed codesize decreasing by 44KB from a baseline build. Experiment 2 showed codesize decreasing by 516KB from a baseline build. So I think this is a good place to start looking at the codesize, at least. On a side note, I realized that PKIX_RETURN does not in fact invoke PKIX_CHECK (which is good, since PKIX_CHECK uses PKIX_RETURN)! Doesn't really change the impact, though. But does mean that reducing the size of PKIX_RETURN is just as important as reducing the size of PKIX_CHECK itself.

<rant> This reminds me of old Unix code from my youth, written by hackers mis-educated that call instructions on a VAX were too costly, so always use macros. Yeesh. Ok, I've been known to overuse macros in the past, so I can speak with the conviction and annoyingly self-righteous attitude of a reformed drunk here :-/. But really, this is just bad macrology. </rant> Go bz! Can we get an assignee for this bug? /be

Boris, PKIX_CHECK is a macro that invokes a function and then checks the return value for an error. If, by "Make PKIX_CHECK a no-op", you literally changed it to emit NO code, not even the function call whose result needs to be checked, then the results are not meaningful. In that case, you eliminated not only error checking code, but the actual functionality being performed. The amount of savings from removing the desired functionality along with the undesired is not an interesting metric, I think.

Nelson, I'm not saying my change was anything resembling correct (and yes, I just changed it to emit no code). I'm just saying that I was correct in my earlier guess that the total size of all the code produced by the PKIX_CHECK macros is a significant part of the codesize increase. I don't expect the codesize savings from working on this macro to be the 516KB I saw by any means (for one thing, if it does become a function the instructions to make the function call will take up some space). Think of the two numbers I cited as the upper and lower bounds of what we can expect from improvements to PKIX_CHECK itself, with no changes elsewhere.

Taking bug. Boris, what kind of size increase did you have in mind for a brand new PKIX implementation ? The 1.2-1.5MB increase in libnss translates to about 400 KB more when zipped, or about one and a half minutes even for a dialup user. I am just trying to get some target metrics to see how we want to deal with this issue.

In reply to comment 5: > 1) Fix issue 4 in that comment in just PKIX_CHECK > Experiment 1 showed codesize decreasing by 44KB from a baseline build. Boris, I think you tried a patch like this one: >+ pkixErrorMsg = PKIX_ErrorText[descNum]; \ > if (pkixErrorCode == PKIX_FATAL_ERROR){ \ >- pkixErrorMsg = PKIX_ErrorText[descNum]; \ > PKIX_RETURN(FATAL); \ > } else { \ >- pkixErrorMsg = PKIX_ErrorText[descNum]; \ > goto cleanup; \ > } \ Is that what you did? If so, I'm surprised it made much difference at all. Any optimizing compiler should have done the equivalent thing. (The optimizing compiler I wrote as a Junior in college did more than that!) So, this makes we wonder: are we using the right compiler optimizer options? If you built NSS with a browser makefile, as you/we sure that it actually did an optimized build of NSS?

There's a lot of obvious room for improvement in pkix_tools.h. The code for logging and tracing seem like obvious candidates to be put into macros of their own, which CONDITIONALLY expand to code, or to nothing. 102 if (pkixLoggersDebugTrace) { \ 103 (pkix_Logger_Check(pkixLoggersDebugTrace, \ 104 funcName, ">>>", pkixType, \ 105 PKIX_LOGGER_LEVEL_TRACE, plContext)); \ 106 } \ 225 if (pkixLoggersErrors) { \ 226 (pkix_Logger_Check(pkixLoggersErrors, \ 227 pkixErrorMsg, NULL, pkixType, \ 228 PKIX_LOGGER_LEVEL_FATALERROR, plContext)); \ 229 } \ 291 if (pkixLoggersDebugTrace) { \ 292 (pkix_Logger_Check(pkixLoggersDebugTrace, \ 293 myFuncName, "<<<", \ 294 pkixType, PKIX_LOGGER_LEVEL_TRACE, plContext)); \ 295 } \ Macro PKIX_RETURN tries to unlock objects and mutexes, even in functions that never possess or lock any objects or mutexes. The code wastes time setting automatic variables to zero before returning. 313 pkixErrorCode = 0; \ 314 return (NULL); \ This 379 (((c) >= 'a')&&((c) <= 'f')) ||\ 380 (((c) >= 'A')&&((c) <= 'F')) is equivalent to (((c)|0x20 >= 'a')&&((c)|0x20 <= 'f')) PKIX_CHECK is absurdly expensive. Better to have at most one copy of that cleanup code per function, (i.e. at a "loser" label) rather than a copy for every function called.

> Boris, what kind of size increase did you have in mind I thinks sayrer's proposal of about the size we got as a result of removing soap (about 300KB) is a good goal to aim for as a start. If we do better, so much the better. But really I'm not the right person to be asking. Brendan might be. Note that size is an issue not just for downloading but also for performance -- less code means better cache coherence, and we have issues enough with caches that it matters. > Boris, I think you tried a patch like this one: Exactly. > Any optimizing compiler should have done the equivalent thing. You'd think. In practice, optimizing compilers don't seem to do a very good job, all things considered... My build is definitely doing an optimized build. Here's typical make command output for one of the pkix files: gcc -o /home/bzbarsky/mozilla/profile/obj-opt/nss/pkixtop/pkix_build.o -c -O2 -fPIC -DLINUX1_2 -Di386 -D_XOPEN_SOURCE -DLINUX2_1 -ansi -Wall -Werror-implicit-function-declaration -pipe -DLINUX -Dlinux -D_POSIX_SOURCE -D_BSD_SOURCE -DHAVE_STRERROR -DXP_UNIX -UDEBUG -DNDEBUG -D_REENTRANT -DNSS_ENABLE_ECC -I/home/bzbarsky/mozilla/profile/obj-opt/dist/include/nspr -I/home/bzbarsky/mozilla/profile/obj-opt/dist/include -I/home/bzbarsky/mozilla/profile/obj-opt/dist/public/nss -I/home/bzbarsky/mozilla/profile/obj-opt/dist/private/nss -I/home/bzbarsky/mozilla/profile/obj-opt/dist/include -I/home/bzbarsky/mozilla/profile/obj-opt/dist/public/dbm pkix_build.c I do wonder what things would look like with -Os instead of -O2... ;)

Boris, Re: comment 12, What is sayrer's proposal ? I see he is cc'ed to this bug but he hasn't participated in it. Brendan, Re: comment 6, It is true that the macros are overused. But the code was certainly not written by hackers, it was written by people with 10 - 25 years experience actually. It is true that there is some redundancy in the macros - more error checks than are sometimes necessary. I don't know about you, but I will take that bloat any day over code with little to no error checking, which is what most hackers and newbies tend to do. That said, we can make an effort to optimize the macros where possible. It will be a lot harder to change the users of the macros due to the size of the libpkix codebase.

(In reply to comment #13) > Boris, > > Re: comment 12, > What is sayrer's proposal ? I see he is cc'ed to this bug but he hasn't > participated in it. I'll help: we should strive to get libpkix as small as we reasonably can, while removing the bad macrology and making the code easier to read for memory safety and other bugs. Has anyone fuzz tested certs? Not that we've found it all that useful, but has anyone run Coverity or similar on the source? > Brendan, > > Re: comment 6, > > It is true that the macros are overused. But the code was certainly not written > by hackers, it was written by people with 10 - 25 years experience actually. Yeah, like me: hackers. I'm a hacker. Nothing pejorative in that term (cf. crackers, now alas falling into disuse). > It is true that there is some redundancy in the macros - more error checks than > are sometimes necessary. I don't know about you, but I will take that bloat any > day over code with little to no error checking, which is what most hackers and > newbies tend to do. That's a false dilemma. Don't even start with me. > That said, we can make an effort to optimize the macros where possible. It will > be a lot harder to change the users of the macros due to the size of the > libpkix codebase. Nothing's that hard here, and we will require significant code size reductions, or else turn off the mystery meat features that depend on libpkix. /be

(In reply to comment #14) > Has anyone fuzz tested certs? Cc'ing Jesse. I meant in particular: has anyone fuzz tested all the fancy new topology Nelson describes in comment 3. /be

Brendan, No, and you cannot do that yet. libpkix is still unfinished business. That's why bug 358785 is not closed yet. Neither is bug 294531 . Even if those 2 bugs were closed, PSM will need new code to take advantage of the new interface before you can do fuzz testing. Kai may have a bug for that too. Right now none of the PKIX code is being exported except if you set NSS_BUILD_TESTS, which will give you a special build that allows the libpkix test programs to build and run. There are a lot of them, and we have fed them lots of certs from various test suites from NIST, but not random certs from random sites. I'm a bit surprised that the linker does not eliminate all the libpkix code from libnss3 right now given that it is still not exposed. See bug 389484 about that.

There's more integration work to do to fully use the new libPKIX everywhere in NSS that now uses the old stuff. But libPKIX is functional now. The timing to get NSS 3.12 alpha into FF3 alpha now is for EV. Our schedule expects to be much father with integration in another month (IIRC).

Julien, I honestly can't recall where sayrer made his suggestion. It might even have been irc. I've covered all there is to it, I think. Nelson, I think there's general agreement that the last possible time for big changes like this is now. It sounds like there's also agreement that we should work on reducing the codesize hit here.

(In reply to comment #18) > Julien, I honestly can't recall where sayrer made his suggestion. It might > even have been irc. I've covered all there is to it, I think. Yes, that is all there is. Julien, it's really not necessary for me to participate here. There's not much I can do to help, and I am not the one to make decisions on the matter. So don't worry about me. :)

Here are some more numbers from the actual mozilla downloads. Here's the breakdown (sizes from actual nightly builds). Windows Firefox .zip (compressed): with NSS 3.11.7 (build 2007-07-23-04): 8682 KB with NSS 3.12 (build 2007-07-23-19): 8908 KB Increase 226KB (2.6%). (Note: Firefox on 5/9 was 8903 Kb, so mozilla has spend the last 2 months whittling down the size). ------------------------------------------------- Other platforms & sizes: Linux Firefox .tar.bz2: with NSS 3.11.7 (build 2007-07-23-04): 8353 KB with NSS 3.12 (build 2007-07-24-04): 8754 KB Increase 401 Kb, 4.8 % Mac Firefox .dmg: with NSS 3.11.7 (build 2007-07-23-04): 15977 KB with NSS 3.12 (build 2007-07-24-04): 16769 KB Increase 792 Kb, 4.9 %

Note that download size and the codesize metric on Tinderbox are very different things, by the way.

More interesting is the comparison between the actual code size increases between Linux and Windows, which seems to indicate Nelson's observation that there may be some 'deficiencies' in the optimizer. It also indicates some of the low hanging fruit won't affect help windows. Library Windows Linux NSS 3.11.7 NSS 3.12 NSS 3.11.7 NSS 3.12 freebl3.dll 200 KB 232 KB 257.5 KB 290.7 KB [Camellia] softokn3.dll 292 KB 208 KB 312.7 KB 238.9 KB [Shared DB] nssdbm3.dll 0 KB 156 KB 0 KB 189.7 KB [Old DB Support] nss3.dll 376 KB 744 KB 455.7 KB 1.4 MB [LibPKIX] ssl3.dll 136 KB 136 KB 163.8 KB 164.2 KB smime3.dll 112 KB 112 KB 142.7 KB 142.7 KB nssckbi.dll 264 KB 288 KB 273.3 KB 293.3 KB total 1.35 MB 1.83 MB 1.57 MB 2.69 MB bob

OK, maybe the difference between windows and Linux is the link optimizer, which Julien showed does a better job of dead code removal on Windows. If that's the case, then getting the Linux code size down is important, otherwise the windows build will increase as we actually add more than the init function back into NSS. bob

Bob, From my tests, the linker on Linux always link all the objects passed on the command-line into the shared object. It doesn't eliminate dead code, at least not with any switches that I have found yet. It's the same on Solaris. So, the Linux and Solaris DSOs will not get any bigger when we add more than the init function. For the record, on solaris x86 with the studio 11 compiler, libnss3.so grows from 911 KB to 2174 KB, or a 138% increase, but that's not a problem for Sun since NSS is bundled with the OS and that's a very small increase compared to the total operating system size. On Windows, the linker is smarter than Linux/Solaris. The Windows DSOs will grow when we export more of libpkix. I have done an experiment by setting BUILD_LIBPKIX_TESTS to export most libpkix symbols, and nss3.dll grew to 1060 KB. I would expect that to be the upper boundary that it will get to for 3.12 when it is released. This is a 175% increase over NSS 3.11, but 1 MB for the NSS library still doesn't seem unreasonable. Boris, Sayrer, Given that the sizes of the DSOs and the increases vary so widely per platform and per compiler, the target goal still remains less than clear. I can shave more than 300 KB off the Solaris shared lib very easily just by changing the level of compiler optimization. The numbers posted by Bob Relyea in comment 20 don't seem that bad. The near 5% increase on Mac and Linux is on the high side but still not horrible. By the time we are done, Windows will probably have a similar percentage increase once some of the currently dead code is no longer eliminated by the linker. So, I think some clarification is still needed on how far we want to go with this. I would say that we should ignore the Windows results right now given that the linker eliminates dead code. Linux, Mac or Solaris are better indicators of the actual code growth we will end up getting with 3.12. I cannot test changes on Mac since I don't have access to one, so the criteria would have to be set on Linux or Solaris. Since Sun doesn't have a problem with the code size on Solaris, the criteria should probably be on Linux. I don't do browser builds, only NSS builds, so I won't be able to measure the % increase for the overall browser. But I can work on the NSS library size. The numbers I am getting here with the compiler we have are slightly different than Bob. On RHEL 4 with gcc 3.4.6, I get 1666 KB for NSS 3.12 and 573 KB for NSS 3.11.7 . That's about a 190% increase. I'll start playing with the macros to see what can be done, but I would like to set a reasonable goal for this library size.

I think the goals discussion should probably happen in mozilla.dev.platform or mozilla.dev.planning, for what it's worth. The only proposal we have so far is that of increasing codesize by no more than 300KB on Linux. That's including all the NSS libraries, not just libnss3. Note that the pkix landing removed some code from other DSOs already; if more can be removed, that's equivalent to shrinking pkix as far as Gecko is concerned. 300KB would correspond to a 25% codesize incrase or so for NSS over here. I have to admit that I'm hesitant to set a goal after which we'll just declare ourselves done even if we could do better... And again, it's not just the shipping zip codesize we care about. We actually care about the unzipped on-disk (or rather to-be-read-into-memory-on-startup) codesize too.

What bz said. It's not as if we don't know of bad macro bloat to fix. We need to step up and commit to real work to avoid increasing net code size, not look for "easy outs". This is an imperative across the project. It's not enforced by any automation, so little things still cause an upward secular trend. But big things are going to get on our radar, and not get off it based on special pleading. /be

This patch reduces the code of code in optimized builds only. It reduces the size of the c code in some functions by more than half. But it is not tested yet, and I have not measured the size of actual generated code. (BTW, seems to me we should be comparing the sizes of text plus data segments, rather than size of object files.) I have a few more ideas. This patch is really just to get a second approximation see how much is due to the redundancy of the macros.

Nelson, the tinderbox test looks at the segment sizes. Anyone who wants to can run it locally. Object file sizes are just faster to look at as a rough approximation.

BTW, another major addition to NSS 3.12 is the Camelia ciphers and suites. This adds support for Japan's new national crypto standards. It adds size across the board to virtually all parts of NSS. This was something that various MoFo representatives seemed to want. I don't think there would have been much motivation to add it to NSS otherwise. I think a reasonable goal at this point is to reduce NSS size by the amount due to coding inefficiency in libPKIX. We don't have a number for that yet, but I think Boris's experiment (comment 5) tells us to expect less than 500K reduction from the current 3.12 numbers on that platform (whatever it was).

We might be able to do better than that depending on what we do with PKIX_RETURN. Then again we might not. The point is, we should try and see what happens.

This gets rid of the memset calls. On X86 this is smaller code.

are these changes being pushed upstream as well? I'd hate for us to have a patched version of libpkix...

Shawn, we *ARE* the upstream for libpkix. We're the stream head.

Adding bug 389781 as a dependency . Building size-optimized alone reduces the code size by a good chunk, about 40% of what we are shooting for on Linux .

Adding bug 389778 as a dependency. Stripping the NSS binaries will significantly reduce file size, though it won't affect code size.

Adding bug 286642 as a dependency. We currently have 4 copies of libnssutil.a linked to various shared libraries. While not every single symbol and object is used by each, a significant code size reduction can be achieved by moving that code to a separate shared library - about 10% of the target here, and that work is in progress for NSS 3.12 .

Note that any time I've cited .so sizes I cited stripped sizes. Unstripped ones are pretty meaningless, of course.

Here is a summary of the work so far . All numbers below are for Linux x86 binaries with gcc 3.4.6 . The total stripped size of all NSS 3.11.7 binaries is 1984 KB . For 3.12, before any changes, it is 3200 KB . I have omitted libsqlite3.so from that total, since that is already in Firefox. With my preliminary patch for libutil, the total goes down to 3101 KB . This is still work-in-progress, mostly pending testing on many platforms. Nelson's attachment 274087 [details] [diff] [review] brings this down to 2749 KB. I'm not sure if the patch is correct yet. That's something that needs to be determined still. Our QA engineer is not actively involved in PKIX yet and it's not automated yet. If it passes the tests, then that means it cut about 40% of the PKIX code. After using gcc -Os to optimize for size, the total of the libraries goes down to 2281 KB. This is < 300 KB increase compared to the 1984 of 3.11, so I think we have met our target. If we consider file size and the fact that 3.11 binaries were not stripped, but we will strip the 3.12 binaries, NSS 3.12 will actually be smaller. I won't be able to do anymore work on this until next tuesday as I only work tuesdays through thursdays currently.

previous patch was missing one file. This passes all.sh

> If we consider file size and the fact that 3.11 binaries were not stripped All libraries shipped with mozilla.org builds are stripped, including NSS.

Comment on attachment 274308 [details] [diff] [review] complete version of previous patch (checked in) requesting review

Comment on attachment 274308 [details] [diff] [review] complete version of previous patch (checked in) r+=relyea. Some random comments. None are necessary to fix to check in: 1) PKIX_DoReturn as a boolean doLogger flag, but logging is only enabled on debug, and PKIX_DoReturn is only called on optimized builds. (Flag isn't a great harm as the test is is in a function, not a macro, and chances are the compilier will optimize out the if(doLogger);) There are a couple of functions that are quite similiar (PKIX_LOG_ERROR/ PKIX_LOG_FATAL_ERROR and PKIX_CHECK and PKIX_CHECK_FATAL_ONLY) which have quite different macros even though they are very similar in functionality. The proposed patch is already better than the original source (It's easier to see the similiarity in the patch than the original). Question.. pkixErrorMsg is an automatic(usually), but we set it and return in some fatal cases in the macros, am I mis understanding the code?

include/pkix_errorstrings.h; new : 1.3; previous : 1.2 include/pkix_pl_system.h; new : 1.3; previous : 1.2 include/pkixt.h; new : 1.3; previous : 1.2 pkix/top/pkix_build.c; new : 1.3; previous : 1.2 pkix/util/manifest.mn; new : 1.4; previous : 1.3 pkix/util/pkix_error.c; new : 1.3; previous : 1.2 pkix/util/pkix_error.h; new : 1.3; previous : 1.2 pkix/util/pkix_errpaths.c; initial : 1.1 pkix/util/pkix_logger.c; new : 1.3; previous : 1.2 pkix/util/pkix_logger.h; new : 1.3; previous : 1.2 pkix/util/pkix_tools.c; new : 1.3; previous : 1.2 pkix/util/pkix_tools.h; new : 1.3; previous : 1.2 pkix_pl_nss/module/pkix_pl_colcertstore.c; new : 1.3; previous : 1.2 pkix_pl_nss/pki/pkix_pl_cert.c; new : 1.4; previous : 1.3 pkix_pl_nss/system/pkix_pl_string.c; new : 1.3; previous : 1.2 In answer to Bob's question: The old macros did a lot of that (setting locals just before returning). The new macros don't do that, IINM. There is one place in the new macros that may appear to be doing that: pkixErrorMsg = PKIX_ErrorText[descNum]; \ PKIX_RETURN(FATAL); \ But actually, the PKIX_RETURN macro does something with it. (Ick, I know) I'm leaving this bug unresolved, because (I think) Julien is going to attach a patch to build libPKIX (and perhaps other parts of NSS) with a compiler option that minimizes size rather than time.

Actually, I assigned bug 389781 to Christophe for the size optimization flags. I'm working on util, bug 286642, and it's unfortunately taking more time than expected. When all these bugs are resolved we can make another alpha tag and give the next NSS 3.12 drop to the client.

The reviewed patch to one file did not get checked in. Now fixed. cmd/libpkix/perf/libpkix_buildthreads.c; new: 1.2; previous: 1.1

This changes have not landed in Mozilla trunk yet, correct?

The mozilla trunk is not yet using a tag that gets any of the changes for this bug.

Julien, is there any part of this bug not yet resolved? If not, please mark it resolved/fixed today.

Taking, since I wrote all the patches for this bug, and committed the only patch for this bug. I don't think this bug needs to wait for bug 286642, although the change for that bug will further reduce code size.

It's OK with me if you want to remove bug 286642 as a dependency.

OK.... so we apparently pulled in this fix on October 11 (bug 397296). Codesize on Linux decreased 300KB or so. The original codesize _increase_ when libpkix landed was 1.2MB. So it looks like we're still 900KB over where we started. Not even close to the 300KB over mark that was claimed in comment 39, much less what we would ideally have (which is 0KB increase). Do you want a new bug? Do you want this bug reopened?

On the other hand, it's possible that in the meantime tinderbox builds switched compilers, with the move to the ref platform, and in the process the old Z data got thrown away so it's a little difficult to do an apples to apples comparison. On my machine I have done no compiler changes. I'll try doing before/after pulls to double-check. I don't have codesighs enabled, sadly, so it'll just be .so sizes for now, which is suboptimal.

OK, I confirmed with my compiler that I'm seeing a drop of about 300KB as well. Initial rise on libpkix landing was over 1MB over here.

How did you build nss? On my build I get the following. Alpha1: bobs-laptop(73) size libnss3.so text data bss dec hex filename 1482901 25228 2484 1510613 170cd5 libnss3.so Alpha2: bobs-laptop(108) size libnss3.so text data bss dec hex filename 879043 25844 2580 907467 dd8cb libnss3.so I'll try firefox builds and see if I see the same difference. bob

Ok it looks like a problem in bug 380781. I'll reopen that bug.

(In reply to comment #57) > Ok it looks like a problem in bug 380781. I'll reopen that bug. That doesn't seem like the bug you meant. Could you post the correct bug nr? Thanks.

Sorry, never mind. It turns out you meant bug 389781.

> How did you build nss? I did whatever it is that client.mk does by default. I get: text data bss dec hex filename 1278585 25712 2484 1306781 13f09d libnss3.so text data bss dec hex filename 985739 26312 2580 1014631 f7b67 libnss3.so for before/after the NSS tag update. And libsoftokn3, libssl3, libfreebl3 all increased in size a bit. Not having -Os happening could in fact do this, yes.

Boris, Re: comment 53, when I wrote comment 39, I was adding up the code size of all NSS shared libraries, not just libnss3.so . Some code is getting moved around with libutil, though I don't think it that made it to alpha2. Some libraries may grow while others may decrease. It's only fair to compare the total size of all libraries. Re: comment 60, I didn't use client.mk for any of my tests, since that is not a part of the NSS build process this file is not maintained by NSS developers. All measurements were made with the NSS standalone build, which is done by going to mozilla/security/nss and doing gmake nss_build_all . Please redo the build this way between the alpha1 and alpha2 tags with the same compiler on your platform. I expect you will see bigger code size drops than you reported.

> Re: comment 53, when I wrote comment 39, I was adding up the code size of all > NSS shared libraries Yes. Comment 53 is talking about total size of all libraries. It really doesn't matter to me what I _could_ build NSS as. What matters is what ends up actually building and shipping as part of Firefox, Seamonkey, Camino, etc. I'll wait for bug 400759 to land and then see what things look like.

Boris, Re: comment 62, Other comments seemed to mention only the size of libnss3.so, so I just wanted to make that clear. And yes, it does matter how you could build it because it helps diagnose where the issue actually lies if there is a difference in the results. As it turns out, somebody else found out it was due to the implementation for bug 389751 which - NSS was checking for the wrong define from the mozilla client build to turn on size optimization.

OK, switching to -Os gets us another 390KB on tinderbox. So at this point on Tinderbox we have: +1.2MB - 300KB - 390KB = +510KB. That's still well short of being able to call this bug "fixed", even if we were to accept the "+300KB" metric for it.

Boris, I did size tests again on Linux x86 (RHEL4) with gcc 3.46 . First, I built from NSPR_4_6_BRANCH and NSS_3_11_BRANCH . I stripped all the binaries. Here was the result : [jp96085@monstre]/home/jp96085/sizes/311 41 % ll total 3958 drwxr-xr-x 2 jp96085 staff 512 Oct 25 21:14 . drwxr-xr-x 4 jp96085 staff 512 Oct 25 21:14 .. -rwxr-xr-x 1 jp96085 staff 293408 Oct 25 21:31 libfreebl3.so -rwxr-xr-x 1 jp96085 staff 200912 Oct 25 21:31 libnspr4.so -rwxr-xr-x 1 jp96085 staff 463428 Oct 25 21:31 libnss3.so -rwxr-xr-x 1 jp96085 staff 279856 Oct 25 21:31 libnssckbi.so -rwxr-xr-x 1 jp96085 staff 15972 Oct 25 21:31 libplc4.so -rwxr-xr-x 1 jp96085 staff 9108 Oct 25 21:31 libplds4.so -rwxr-xr-x 1 jp96085 staff 146152 Oct 25 21:31 libsmime3.so -rwxr-xr-x 1 jp96085 staff 361356 Oct 25 21:31 libsoftokn3.so -rwxr-xr-x 1 jp96085 staff 171812 Oct 25 21:31 libssl3.so [jp96085@monstre]/home/jp96085/sizes/311 42 % du 1978 . This is very close to the 1984 KB I quoted in comment 39 . So that was the baseline. Then, I built the trunk of NSPR and NSS. I didn't use the alpha 2 tag, since not all the size optimization work went into it. Specifically, the fix for bug 286642 was taken into account in comment 39 ("with my preliminary patch for libutil"), but that didn't make it to the alpha 2 . I built NSS with OPT_CODE_SIZE=1, and I stripped all the binaries. Here was the result : [jp96085@monstre]/home/jp96085/sizes/trunk 46 % ll total 4702 drwxr-xr-x 2 jp96085 staff 512 Oct 25 21:38 . drwxr-xr-x 4 jp96085 staff 512 Oct 25 21:14 .. -rwxr-xr-x 1 jp96085 staff 325352 Oct 25 21:38 libfreebl3.so -rwxr-xr-x 1 jp96085 staff 200944 Oct 25 21:38 libnspr4.so -rwxr-xr-x 1 jp96085 staff 844036 Oct 25 21:38 libnss3.so -rwxr-xr-x 1 jp96085 staff 270332 Oct 25 21:38 libnssckbi.so -rwxr-xr-x 1 jp96085 staff 116496 Oct 25 21:38 libnssdbm3.so -rwxr-xr-x 1 jp96085 staff 79624 Oct 25 21:38 libnssutil3.so -rwxr-xr-x 1 jp96085 staff 13876 Oct 25 21:38 libplc4.so -rwxr-xr-x 1 jp96085 staff 9108 Oct 25 21:38 libplds4.so -rwxr-xr-x 1 jp96085 staff 121716 Oct 25 21:38 libsmime3.so -rwxr-xr-x 1 jp96085 staff 185140 Oct 25 21:38 libsoftokn3.so -rwxr-xr-x 1 jp96085 staff 143572 Oct 25 21:38 libssl3.so [jp96085@monstre]/home/jp96085/sizes/trunk 47 % du 2350 . Note that I deleted libsqlite3 from the trunk directory to make the comparison fair, since it is already part of firefox. This is a little higher than the 2281 KB I had listed in comment 39, but it is not far. Several months of development on the NSS trunk have happened since, so this could account for the increase of 69 KB since my last size test. So, as of today, the difference between the 3.11 branch and the NSS trunk for Linux x86 is 2350 - 1978 = 372 KB. It should be noted that NSPR is still not being built optimized for size in my experiment. I don't know if it's desirable to do that or not, but we could probably shave a few KB there, maybe 10 to 20.

> I didn't use the alpha 2 tag Ah, ok. I'd been under the impression that the tag we're pulling right now (which is alpha 2) had picked up all the work for this bug. Please let me know once there is an NSS tag which _has_ picked up all the codesize work, so that we can update trunk Gecko to pull that tag and see what things look like. > this could account for the increase of 69 KB since my last size test. That's a pretty noticeable increase too, to be honest. Do we know why it's increased? Maybe we need a HEAD NSS tinderbox with codesighs running on it so we catch these things when they land, not when the tag is updated? > It should be noted that NSPR is still not being built optimized for size That's an interesting question. In the Gecko build we build NSPR ourselves, so I'm not sure what compiler flags it ends up with... Anyone know?

I'm not sure exactly what accounts for the extra 69 KB. One would have to review 3 months of checkins on the NSS trunk. There is nothing that comes to mind. It could be that the util patch wins slightly less than it originally did in my test due to some issues that came up during implementation. I haven't done measurements just prior to the checkin, but the alpha2 was cut pretty close in time before that checkin. We have a trunk NSS tinderbox, but currently it doesn't display code size. It also builds things slightly differently than mozilla does, in particular, it doesn't build with code size optimization enabled. But a display of code size could still be added to it, and you could see relative increases after checkins. Re: NSPR, you might already be building it optimized for size, but we don't. I don't have a gecko browser build log to look at to check.

> One would have to review 3 months of checkins on the NSS trunk. Right. As we do when the tag gets updated... hence my suggestion for a tinderbox test. > We have a trunk NSS tinderbox, but currently it doesn't display code size. Can we get that changed? That way NSS developers could see when they increase codesize.... > It also builds things slightly differently than mozilla does From what I've seen, that tends to affect the exact numbers, but not the relative numbers. That is, a N% codesize increase is usually about N% of the baseline pretty much no matter what compiler flags are being used. Except for pathological cases, of course, which are somewhat rare. Put another way, most new code is about as easy or hard to optimize for size as the already-existing code.

With the landing of bug 399590, we dropped another 90KB. So overall we're at +420KB for this code. I believe all the proposed size improvements have now landed, right? Since this is nowhere close to even the +300KB bare-minimum metric, I don't see how we can call this bug fixed. Reopening.

I built NSS_3_12_BETA1 according to the same methodology I used before, on RHEL3 with gcc 3.4.6, optimized and with OPT_CODE_SIZE=1, stripped . The resulting binaries are : -rwxr-xr-x 1 jp96085 staff 322536 Jan 22 2008 libfreebl3.so -rwxr-xr-x 1 jp96085 staff 209600 Jan 22 2008 libnspr4.so -rwxr-xr-x 1 jp96085 staff 914632 Jan 22 2008 libnss3.so -rwxr-xr-x 1 jp96085 staff 289240 Jan 22 2008 libnssckbi.so -rwxr-xr-x 1 jp96085 staff 119008 Jan 22 2008 libnssdbm3.so -rwxr-xr-x 1 jp96085 staff 82192 Jan 22 2008 libnssutil3.so -rwxr-xr-x 1 jp96085 staff 14980 Jan 22 2008 libplc4.so -rwxr-xr-x 1 jp96085 staff 9564 Jan 22 2008 libplds4.so -rwxr-xr-x 1 jp96085 staff 126284 Jan 22 2008 libsmime3.so -rwxr-xr-x 1 jp96085 staff 189480 Jan 22 2008 libsoftokn3.so -rwxr-xr-x 1 jp96085 staff 148192 Jan 22 2008 libssl3.so % du 2475 . So we picked up 125 KB since my last measurement on 10/26 . It seems that gains were spread among all libraries. Only libfreebl3 shrunk by 3 KB since then. nssckbi grew by 19 KB. Are we expected to reduce the NSS code size each time that we add new root certs ?

I don't think anyone's demanding the root cert thing. I certainly am not. What about the other 100+KB?

Unsetting target milestone in unresolved bugs whose targets have passed.

This bug is as fixed as it is ever going to be. If you're prefer WONTFIX, be my guest.

I think it's more like WONTFIX, but bz's question from comment 71 seems like it's worth an answer, since codesize is used as an argument against implementing requested capabilities in other bugs! :-)