Closed Bug 1253436 Opened 9 years ago Closed 9 years ago

Write a machine readable file containing moz.build derived binaries metadata

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox48 fixed)

RESOLVED FIXED
mozilla48
Tracking Status
firefox48 --- fixed

People

(Reporter: gps, Assigned: gps)

References

(Blocks 2 open bugs)

Details

Attachments

(3 files)

In bug 1253110 we'll be producing archives with binaries files. Instead of hard-coding a list of binaries (like we do with artifact builds today), let's derive the list of binaries from moz.build. The first step is to produce a machine readable file containing binaries metadata.
Now when we print instances in a debugger, we'll see something like: <StaticLibrary: other-licenses/snappy/libother-licenses_snappy.a> <StaticLibrary: toolkit/library/StaticXULComponentsEnd/libStaticXULComponentsEnd.a> <SharedLibrary: toolkit/library/XUL> <StaticLibrary: toolkit/library/libxul_s.a> <HostProgram: config/nsinstall_real> <Program: memory/replace/logalloc/replay/logalloc-replay> <SimpleProgram: mfbt/tests/TestArrayUtils> ... instead of the the class name and memory address. Review commit: https://reviewboard.mozilla.org/r/37995/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/37995/
Attachment #8726447 - Flags: review?(mh+mozilla)
This will make it easier for binaries only archive generation to consult the list of binaries that are relevant to packaging. This does overlap somewhat with compile databases. Perhaps someday the logic could converge. While I was here, I cleaned up writing of the all-tests.json file to avoid an intermediate string variable. Review commit: https://reviewboard.mozilla.org/r/37997/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/37997/
Attachment #8726448 - Flags: review?(mh+mozilla)
It just occurred to me that we could potentially use this code for updating the dist/bin install manifest. We could also potentially write out make rules to create binaries in their final location under dist/bin. That is scope bloat, however.
Comment on attachment 8726448 [details] MozReview Request: Bug 1253436 - Write out a machine readable file with binaries metadata; r?glandium Review request updated; see interdiff: https://reviewboard.mozilla.org/r/37997/diff/1-2/
Comment on attachment 8726446 [details] MozReview Request: Bug 1253436 - Remove unused imports; r=glandium https://reviewboard.mozilla.org/r/37993/#review35273
Attachment #8726446 - Flags: review?(mh+mozilla) → review+
Comment on attachment 8726447 [details] MozReview Request: Bug 1253436 - Add __repr__ to BaseLibrary and BaseProgram; r=glandium https://reviewboard.mozilla.org/r/37995/#review35275
Attachment #8726447 - Flags: review?(mh+mozilla) → review+
Comment on attachment 8726448 [details] MozReview Request: Bug 1253436 - Write out a machine readable file with binaries metadata; r?glandium https://reviewboard.mozilla.org/r/37997/#review35277 ::: python/mozbuild/mozbuild/backend/common.py:204 (Diff revision 2) > + def to_dict(self): > + SHARED_ATTRS = ('basename', 'lib_name', 'import_name', 'soname', > + 'install_target', 'relobjdir') > + shared = [] > + for l in self.shared_libraries: > + shared.append({k: getattr(l, k) for k in SHARED_ATTRS}) > + > + PROGRAM_ATTRS = ('program', 'KIND', 'install_target', 'relobjdir') > + programs = [] > + for p in self.programs: > + programs.append({k.lower(): getattr(p, k) for k in PROGRAM_ATTRS}) > + > + return { > + 'shared_libraries': shared, > + 'programs': programs, > + } I'd rather use a JSONEncoder subclass, that could be repurposed later.
Attachment #8726448 - Flags: review?(mh+mozilla)
I'd love to figure out a more general solution here to capture the parsed moz.build data for use in scripts/mach commands/etc instead of continuing to accumulate one-off json files. That's not to say we shouldn't land this patch, since it's useful, but it would be great to figure out a better long-term plan.
Comment on attachment 8726446 [details] MozReview Request: Bug 1253436 - Remove unused imports; r=glandium Review request updated; see interdiff: https://reviewboard.mozilla.org/r/37993/diff/1-2/
Attachment #8726446 - Attachment description: MozReview Request: Bug 1253436 - Remove unused imports; r?glandium → MozReview Request: Bug 1253436 - Remove unused imports; r=glandium
Comment on attachment 8726447 [details] MozReview Request: Bug 1253436 - Add __repr__ to BaseLibrary and BaseProgram; r=glandium Review request updated; see interdiff: https://reviewboard.mozilla.org/r/37995/diff/1-2/
Attachment #8726447 - Attachment description: MozReview Request: Bug 1253436 - Add __repr__ to BaseLibrary and BaseProgram; r?glandium → MozReview Request: Bug 1253436 - Add __repr__ to BaseLibrary and BaseProgram; r=glandium
Comment on attachment 8726448 [details] MozReview Request: Bug 1253436 - Write out a machine readable file with binaries metadata; r?glandium Review request updated; see interdiff: https://reviewboard.mozilla.org/r/37997/diff/2-3/
Attachment #8726448 - Flags: review?(mh+mozilla)
Attachment #8726448 - Flags: review?(mh+mozilla) → review+
Comment on attachment 8726448 [details] MozReview Request: Bug 1253436 - Write out a machine readable file with binaries metadata; r?glandium https://reviewboard.mozilla.org/r/37997/#review36461 > In the spirit of perfect is the enemy of done, I think this should be deferred to a follow-up bug. Please file that followup bug, otherwise, it's never going to happen. ::: python/mozbuild/mozbuild/frontend/data.py:379 (Diff revision 3) > + def to_dict(self): > + return {k: getattr(self, k) for k in self.DICT_ATTRS} You might as well move this method to the base class. It will fail on all classes with no DICT_ATTRS, which is fine.
Blocks: 1256525
Flags: needinfo?(gps)
Fixed the case of 'kind' vs 'KIND' to unbust it.
Flags: needinfo?(gps)
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
Blocks: 1259789
No longer depends on: 1259789
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: