In bug 540627 I added an environment variable, MINIDUMP_SAVE_PATH, to the test harness code so we can save minidumps that are generated during test runs. I'd like to have this set for the test runs on tinderbox so that developers can get access to minidumps from test runs. This will be a manual process currently, involving asking whoever's on build duty to fetch the minidump from the machine. (The minidump filename will be printed in the log.)

From luke1 in irc, this is to help track down an intermittent Md assertion that Md unittest suite is failing out "a few times a day for the last 3 days, only on osx10.5". 1) We're close to diskspace limits on most slaves, so have to ask how big is the minidump file that is generated? If there a minidump file already present, would a new minidump file overwrite, or somehow get versioned beside the existing? 2) This will need a matching change in our slaves to set the environment variable when running a unittest job. Need patch.

1) Each minidump is ~50k generally. They won't be overwritten with this patch, they'll just accumulate. 2) Yes, this bug is for precisely that.

Ted, a couple questions are these text files? would echoing them into the build log be satisfactory? If they are binary files, would base64 encoding+echoing be satisfactory? If they are echoed, do we need to keep the files around? If we can echo them into the logs, I can take a look at this today

No, these are binary minidump files. We already dump information from the minidump using minidump-stackwalk, but the minidump file contains more information and can be opened up directly in Visual Studio.

is base64 encoding them acceptable?

Is this just about adding a new environment variable? To which unit test builders? packaged tests? What is the value of this variable to be? I don't know enough so I might be asking dumb questions.

(In reply to comment #5) > is base64 encoding them acceptable? (In reply to comment #6) > Is this just about adding a new environment variable? > To which unit test builders? packaged tests? > What is the value of this variable to be? > > I don't know enough so I might be asking dumb questions. bsmedberg/ted: gentle ping?

(In reply to comment #6) > Is this just about adding a new environment variable? > To which unit test builders? packaged tests? > What is the value of this variable to be? This would need to be a new environment variable for packaged unittest runs, set to point to a directory where the test harness could store minidumps produced during the test run, so that if a developer wants to take a look at a minidump from a test failure, someone from RelEng could easily locate the minidump and provide it. This isn't immediately blocking anything, it turns out we got Luke the info he needed in another way, but I still think it'd be useful. I'm not sure if base64 encoding them would be very useful, it seems pretty awkward.

Do we need to ensure the directory exists first?

The code currently assumes that, yes, but if it'd be easier I could fix it to attempt to create the dir first.

Will test this in staging. After checking with Ted, this needs to point to a directory that exists (or add a build step creating the dir/checking for dir) beforehand. Also it will need a cleanup job so that too many logs don't accumulate and take up space.

is it possible for whatever is doing the minidump to check and clean up after itself?

Sorry, I am not getting time for this. Putting back in the pool.

Assigned during triage; if you think this is better assigned to someone else, please swap with them for another old bug.

So I think this is a simple change to the env=.. parameter of a BuildStep in the buildbot-configs repo. But even as a relenger that was out of my scope, and is certainly out of my scope as an IT person.

I want to put these minidumps in ${basedir}/minidumps on the slaves - does that make sense?

What's basedir, the build directory? That seems fine to me.

(In reply to comment #17) > What's basedir, the build directory? That seems fine to me. Almost, e.g.: basedir: /build/slave/m-cen-lnx builddir: /build/slave/m-cen-lnx/build Should still get cleaned up by clobberer when required.

Comment on attachment 557839 [details] [diff] [review] Set MINIDUMP_SAVE_PATH wherever MINIDUMP_STACKWALK is set Looks good

Comment on attachment 557839 [details] [diff] [review] Set MINIDUMP_SAVE_PATH wherever MINIDUMP_STACKWALK is set https://hg.mozilla.org/build/buildbotcustom/rev/485edb076976

landed with 2011-09-06 reconfig