User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0 Build ID: 20161024062541 Steps to reproduce: Build firefox from source using changeset: 320921:9c6b3b3a119b Enviroment: Ubuntu 16.04 64-bit ./mach build ./mach package (fails) Actual results: Fails to create a package (...) 0:08.44 Duplicates 126280 bytes: 0:08.44 firefox-nightly 0:08.44 firefox-nightly-bin 0:08.44 0:08.44 WARNING: Found 35 duplicated files taking 208714 bytes (uncompressed) 0:08.44 ERROR: The following duplicated files are not allowed: 0:08.44 firefox-nightly 0:08.44 firefox-nightly-bin 0:08.44 /home/averissimo/no-backup/firefox-wrapper/firefox/toolkit/mozapps/installer/packager.mk:41: recipe for target 'stage-package' failed 0:08.44 make[3]: *** [stage-package] Error 1 0:08.44 make[3]: Leaving directory '/home/averissimo/no-backup/firefox-wrapper/firefox/obj-x86_64-pc-linux-gnu/browser/installer' 0:08.44 /home/averissimo/no-backup/firefox-wrapper/firefox/toolkit/mozapps/installer/packager.mk:97: recipe for target 'make-package' failed 0:08.44 make[2]: *** [make-package] Error 2 0:08.44 make[2]: Leaving directory '/home/averissimo/no-backup/firefox-wrapper/firefox/obj-x86_64-pc-linux-gnu/browser/installer' 0:08.44 /home/averissimo/no-backup/firefox-wrapper/firefox/config/rules.mk:523: recipe for target 'default' failed 0:08.44 make[1]: *** [default] Error 2 0:08.44 make[1]: Leaving directory '/home/averissimo/no-backup/firefox-wrapper/firefox/obj-x86_64-pc-linux-gnu/browser/installer' 0:08.44 /home/averissimo/no-backup/firefox-wrapper/firefox/browser/build.mk:9: recipe for target 'package' failed 0:08.44 make: *** [package] Error 2 0:08.44 make: Leaving directory '/home/averissimo/no-backup/firefox-wrapper/firefox/obj-x86_64-pc-linux-gnu' Expected results: Create a tar package for firefox nightly with custom binary. It works without: --with-app-name=firefox-nightly

This is a regression from bug 1303184. catlee: can you please take a look? Somewhere you'll need to plug in MOZ_APP_NAME for the paths to the binaries.

Thanks for the tip on the right direction. This is solved by diff below. I suck at python and even more in firefox, so this solution is by no means perfect or global. diff -r f13e90d496cf toolkit/mozapps/installer/find-dupes.py --- a/toolkit/mozapps/installer/find-dupes.py Mon Nov 07 20:38:29 2016 -0800 +++ b/toolkit/mozapps/installer/find-dupes.py Tue Nov 08 14:51:32 2016 +0000 @@ -9,6 +9,7 @@ from mozpack.files import DeflatedFile from collections import OrderedDict import argparse +import buildconfig ''' Find files duplicated in a given packaged directory, independently of its @@ -110,7 +111,14 @@ for filename in args.dupes_files: with open(filename) as dupes_file: allowed_dupes.extend([line.partition('#')[0].rstrip() for line in dupes_file]) - + ''' + Add --app-name to duplicates + ''' + allowed_dupes.extend([buildconfig.substs['MOZ_APP_NAME'].rstrip()]) + allowed_dupes.extend([buildconfig.substs['MOZ_APP_NAME'].rstrip() + '-bin']) + ''' + call on find_dupes + ''' find_dupes(args.directory, bail=not args.warning, allowed_dupes=allowed_dupes) if __name__ == "__main__":

I've been looking at a very similar approach. I havent' been able to verify if it actually works or not though :) diff --git a/browser/installer/Makefile.in b/browser/installer/Makefile.in --- a/browser/installer/Makefile.in +++ b/browser/installer/Makefile.in @@ -5,17 +5,17 @@ STANDALONE_MAKEFILE := 1 DIST_SUBDIR := browser include $(topsrcdir)/config/rules.mk MOZ_PKG_REMOVALS = $(srcdir)/removed-files.in MOZ_PKG_MANIFEST = $(srcdir)/package-manifest.in -MOZ_PKG_DUPEFLAGS = -f $(srcdir)/allowed-dupes.mn +MOZ_PKG_DUPEFLAGS = -f $(srcdir)/allowed-dupes.mn --app-name $(MOZ_APP_NAME) # Some files have been already bundled with xulrunner ifndef MOZ_MULET MOZ_PKG_FATAL_WARNINGS = 1 else DEFINES += -DMOZ_MULET endif diff --git a/toolkit/mozapps/installer/find-dupes.py b/toolkit/mozapps/installer/find-dupes.py --- a/toolkit/mozapps/installer/find-dupes.py +++ b/toolkit/mozapps/installer/find-dupes.py @@ -96,22 +96,28 @@ def find_dupes(source, allowed_dupes, ba def main(): parser = argparse.ArgumentParser(description='Find duplicate files in directory.') parser.add_argument('--warning', '-w', action='store_true', help='Only warn about duplicates, do not exit with an error') parser.add_argument('--file', '-f', action='append', dest='dupes_files', default=[], help='Add exceptions to the duplicate list from this file') + parser.add_argument('--app-name', dest='app_name', + help='Application name') parser.add_argument('directory', help='The directory to check for duplicates in') args = parser.parse_args() allowed_dupes = [] for filename in args.dupes_files: with open(filename) as dupes_file: allowed_dupes.extend([line.partition('#')[0].rstrip() for line in dupes_file]) + if args.app_name: + # TODO: Remove firefox/firefox-bin? + allowed_dupes.extend([args.app_name, '{}-bin'.format(args.app_name)]) + find_dupes(args.directory, bail=not args.warning, allowed_dupes=allowed_dupes) if __name__ == "__main__": main()

I've also hit this issue on macOS with FIREFOX_BETA_52_BASE tag in mozilla-beta.

Chris, can you make an actual patch of this and r? a build peer?

Comment on attachment 8844808 [details] Bug 1315309 - Preprocess find-dupes exception list. https://reviewboard.mozilla.org/r/118134/#review120202

Pushed by mh@glandium.org: https://hg.mozilla.org/integration/autoland/rev/af4fdc8f073a Preprocess find-dupes exception list. r=mshal

Comment on attachment 8844808 [details] Bug 1315309 - Preprocess find-dupes exception list. Approval Request Comment [Feature/Bug causing the regression]: Regression from bug 1303184 [User impact if declined]: Can't build with --with-app-name. This is an option that tends to be used by organisations that prefer to stay on ESR, thus the a? for ESR too. [Is this code covered by automated tests?]: No [Has the fix been verified in Nightly?]: No, but verified on Debian firefox-esr packages. [Needs manual test from QE? If yes, steps to reproduce]: N/A [List of other uplifts needed for the feature/fix]: N/A [Is the change risky?]: No [Why is the change risky/not risky?]: Essentially, it's NPOTB. The only thing that could go wrong with a patch to the exception list is that duplicate files are not marked as allowed anymore, which would break the build, which didn't happen. [String changes made/needed]: N/A

Comment on attachment 8844808 [details] Bug 1315309 - Preprocess find-dupes exception list. Fix a build issue. Aurora54+ & Beta53+.

has problems landing in beta like: grafting 405976:b1974c6495e8 "Bug 1315309 - Preprocess find-dupes exception list. r=mshal a=gchang" merging browser/installer/allowed-dupes.mn merging toolkit/mozapps/installer/packager.mk warning: conflicts while merging toolkit/mozapps/installer/packager.mk! (edit, then use 'hg resolve --mark') abort: unresolved conflicts, can't continue (use 'hg resolve' and 'hg graft --continue') could you take a look, thanks!

There was a trivial conflict. https://hg.mozilla.org/releases/mozilla-beta/rev/2c6189b62427a436363233adcd0a43743604fef5

Comment on attachment 8844808 [details] Bug 1315309 - Preprocess find-dupes exception list. fix packaging issue when using --with-app-name, esr52+