Tl;dr: is it possible to create a manifest with artifact names, that the build system then uses to name its output files? Longer: In the taskcluster graph, we're using simple names because they're predictable. We can declare the simple names in the task definitions in the graph, and we have a beetmover (bit mover) task at the end of the graph to rename the simple names to pretty names and push to s3. However, we still need to guess the simple names when we're dealing with multiple platforms: is the installer target.exe or target.zip? Or is it a different platform, target.tar.bz2 or target.dmg? And beetmover currently has its own logic about how to rename the simple names to pretty names (see bug 1331141 with how we might deal with that in the shorter term.) We're thinking a declarative manifest may be the solution, mapping the role of an artifact with its name. We can use that manifest to generate the taskcluster graph, and the build system could use it to know what to name its artifacts. If the names are pretty names, beetmover doesn't have to perform any name transforms at all, and can just push pretty-named artifacts to the proper s3 bucket.

dupe of Bug 1330648 ? I'm going to clone as yours is probably in the better component with more verbose description.

The package filenames are all computed in package-name.mk: https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/package-name.mk But they only depend on the target OS, CPU and application version. We could easily compute them all up-front for a given build. The application version lives in a file in-tree: https://dxr.mozilla.org/mozilla-central/source/browser/config/version.txt https://dxr.mozilla.org/mozilla-central/rev/3e275d37a06236981bff399b7d7aa0646be3fee7/old-configure.in#812

Hm, could we define the names in templatized version? win32 installer -> "Firefox {version}.exe" or "%(appname)s %(version)s.exe" ? That could allow for easier generation outside of configure.

If you can figure out a sensible way to define those templates I'd be fine with that.

n.b., all the `MOZ_PKG_PRETTYNAMES` stuff is being removed in bug 1329355.

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #6) > n.b., all the `MOZ_PKG_PRETTYNAMES` stuff is being removed in bug 1329355. Cool. That means we could continue with simple names for everything, and use beetmover to create pretty names, as we will be doing in TC nightly graphs. Bug 1331141 should allow for a single commit in-tree to handle a new artifact, though one would still have to change at least 3 places (add the simple named artifact in the build system; add the simple named artifact in the task graph; add the simple name -> pretty name rule in the beetmover manifests). Some final multiplatform tasks would still need some sort of regex or map to figure out which files to look at per platform (e.g., is the shipping file for platform X a tarball, or exe, or apk, or ?) A pre-declared manifest template would still help streamline those cases.

Is the manifest needed before the build runs? If not, we could essentially generate this from the UPLOAD_FILES Makefile variable. We sort-of already do this in mach_build_properties.json by defining the packageFilename property, so that we can easily set the packageUrl without trying to guess which file is the package in a regex. Right now we do that by passing in a '--package $(PACKAGE)' flag to upload.py, but we could extend that to do more than just the package filename.

Here's an example of what I mean - this produces properties like "package": "target.tar.bz2" in a Linux TC build, or "package": "firefox-53.0a1.en-US.win64.zip", "installer": "firefox-53.0a1.en-US.win64.installer.exe" in a Windows buildbot build. Then you could always just grab the artifact that's tagged "package", and that corresponds to what we know as the $(PACKAGE) in build config land. I just did the package and installer as an example, but we could tag anything that goes into UPLOAD_FILES. If we're generating a separate manifest somewhere, we'll want to use that in upload.py instead of the upload-files.mk logic so that we're not duplicating things.

(In reply to Michael Shal [:mshal] from comment #8) > Is the manifest needed before the build runs? Yes, the decision task will need this while generating the graph, which is before the build runs. If we have to run a script, we'd have to run it for every platform / build flavor combo that is in the graph, which might get expensive or complex. This is largely why I'm leaning towards a manifest with templatized names.