Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Description • 6 years ago

I recently updated to m-c tip and did a |./mach build && ./mach package| for Fennec. This was working fine previously, but now the |./mach package| step fails because it can't find dist/bin/features/webcompat@mozilla.org/chrome/content/data/ua_overrides.jsm

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 1 • 6 years ago

Full output:

kats@kgupta-pc mozilla-fennec$ ./mach package
 0:00.25 /usr/bin/make -C . -j4 -s -w package
 0:00.26 make: Entering directory '/home/kats/zspace/mozilla-fennec/obj-android-debug'
 0:00.86 make[1]: Entering directory '/home/kats/zspace/mozilla-fennec/obj-android-debug/mobile/android/installer'
 0:01.16 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:71: Missing file(s): bin/libnssdbm3.so
 0:01.17 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:104: Missing file(s): bin/blocklist.xml
 0:01.17 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:109: Missing file(s): bin/components/components.manifest
 0:01.21 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:200: Missing file(s): bin/components/nsINIProcessor.manifest
 0:01.21 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:201: Missing file(s): bin/components/nsINIProcessor.js
 0:01.21 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:216: Missing file(s): bin/components/SystemMessageInternal.js
 0:01.21 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:217: Missing file(s): bin/components/SystemMessageManager.js
 0:01.21 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:218: Missing file(s): bin/components/SystemMessageCache.js
 0:01.21 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:219: Missing file(s): bin/components/SystemMessageManager.manifest
 0:01.75 Warning: /home/kats/zspace/mozilla-fennec/mobile/android/installer/package-manifest.in:295: Missing file(s): bin/defaults/pref/channel-prefs.js
 0:01.98 Traceback (most recent call last):
 0:01.98   File "/home/kats/zspace/mozilla-fennec/toolkit/mozapps/installer/packager.py", line 340, in <module>
 0:01.98     main()
 0:01.98   File "/home/kats/zspace/mozilla-fennec/toolkit/mozapps/installer/packager.py", line 334, in main
 0:01.98     copier.copy(args.destination)
 0:01.98   File "/home/kats/zspace/mozilla-fennec/python/mozbuild/mozpack/copier.py", line 430, in copy
 0:01.98     copy_results.append((destfile, f.copy(destfile, skip_if_older)))
 0:01.98   File "/home/kats/zspace/mozilla-fennec/python/mozbuild/mozpack/copier.py", line 603, in copy
 0:01.98     file.copy(deflater, skip_if_older)
 0:01.98   File "/home/kats/zspace/mozilla-fennec/python/mozbuild/mozpack/files.py", line 194, in copy
 0:01.98     shutil.copyfileobj(self.open(), dest)
 0:01.98   File "/home/kats/zspace/mozilla-fennec/python/mozbuild/mozpack/files.py", line 222, in open
 0:01.98     return open(self.path, 'rb')
 0:01.98 IOError: [Errno 2] No such file or directory: '../../../dist/bin/features/webcompat@mozilla.org/chrome/content/data/ua_overrides.jsm'
 0:01.99 /home/kats/zspace/mozilla-fennec/toolkit/mozapps/installer/packager.mk:22: recipe for target 'stage-package' failed
 0:01.99 make[1]: *** [stage-package] Error 1
 0:01.99 make[1]: Leaving directory '/home/kats/zspace/mozilla-fennec/obj-android-debug/mobile/android/installer'
 0:01.99 /home/kats/zspace/mozilla-fennec/mobile/android/build.mk:11: recipe for target 'package' failed
 0:01.99 make: *** [package] Error 2
 0:01.99 make: Leaving directory '/home/kats/zspace/mozilla-fennec/obj-android-debug'

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 2 • 6 years ago

Possibly regression from bug 1451484?

	Andrew Swan [:aswan]
	Comment 3 • 6 years ago

Does it work after a clobber?

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 4 • 6 years ago

Yes, it does.

	(not currently active) Ted Mielczarek
	Comment 5 • 6 years ago

This is unfortunate. It'd be worth digging in to find out what the root cause is.

	(not currently active) Ted Mielczarek
	Comment 6 • 6 years ago

OK, I think I see the issue. The Android package manifest has a wildcard for `features/*`:
https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/mobile/android/installer/package-manifest.in#268

Without a clobber there will be broken symlinks to the previous structure before the patch in dist/bin/features/*, and they will get added to the list of things to package but then the copier will fail to copy them because the symlinks are broken.

We'd probably need to track changes to the contents of install manifests in dep builds and remove outdated entries to fix this.

	Jan Henning [:JanH]
	Comment 7 • 6 years ago

I've noticed similar failures in the past for what I think were XBL binding removals or something like that, but never got around to filing a bug for that.

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 8 • 6 years ago

I hit this again, for the removal of the progressmeter stuff in bug 1499947. Same deal where ./mach package on Fennec failed.

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 9 • 6 years ago

This is the output, in case it's helpful:

$ ./mach package
 0:00.20 /usr/bin/make -C . -j4 -s -w package
 0:00.20 make: Entering directory '/home/kats/zspace/gecko-fennec/obj-android-debug'
 0:00.23 make[1]: Entering directory '/home/kats/zspace/gecko-fennec/obj-android-debug/mobile/android/installer'
 0:00.41 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:71: Missing file(s): bin/libnssdbm3.so
 0:00.41 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:104: Missing file(s): bin/blocklist.xml
 0:00.41 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:109: Missing file(s): bin/components/components.manifest
 0:00.43 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:198: Missing file(s): bin/components/nsINIProcessor.manifest
 0:00.43 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:199: Missing file(s): bin/components/nsINIProcessor.js
 0:00.43 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:214: Missing file(s): bin/components/SystemMessageInternal.js
 0:00.43 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:215: Missing file(s): bin/components/SystemMessageManager.js
 0:00.43 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:216: Missing file(s): bin/components/SystemMessageCache.js
 0:00.43 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:217: Missing file(s): bin/components/SystemMessageManager.manifest
 0:00.53 Warning: /home/kats/zspace/gecko-fennec/mobile/android/installer/package-manifest.in:293: Missing file(s): bin/defaults/pref/channel-prefs.js
 0:00.92 Traceback (most recent call last):
 0:00.92   File "/home/kats/zspace/gecko-fennec/toolkit/mozapps/installer/packager.py", line 340, in <module>
 0:00.92     main()
 0:00.92   File "/home/kats/zspace/gecko-fennec/toolkit/mozapps/installer/packager.py", line 334, in main
 0:00.92     copier.copy(args.destination)
 0:00.92   File "/home/kats/zspace/gecko-fennec/python/mozbuild/mozpack/copier.py", line 430, in copy
 0:00.92     copy_results.append((destfile, f.copy(destfile, skip_if_older)))
 0:00.92   File "/home/kats/zspace/gecko-fennec/python/mozbuild/mozpack/copier.py", line 603, in copy
 0:00.92     file.copy(deflater, skip_if_older)
 0:00.92   File "/home/kats/zspace/gecko-fennec/python/mozbuild/mozpack/files.py", line 194, in copy
 0:00.92     shutil.copyfileobj(self.open(), dest)
 0:00.92   File "/home/kats/zspace/gecko-fennec/python/mozbuild/mozpack/files.py", line 222, in open
 0:00.92     return open(self.path, 'rb')
 0:00.92 IOError: [Errno 2] No such file or directory: '../../../dist/bin/chrome/toolkit/content/global/elements/progressmeter.js'
 0:00.94 /home/kats/zspace/gecko-fennec/toolkit/mozapps/installer/packager.mk:22: recipe for target 'stage-package' failed
 0:00.94 make[1]: *** [stage-package] Error 1
 0:00.94 make[1]: Leaving directory '/home/kats/zspace/gecko-fennec/obj-android-debug/mobile/android/installer'
 0:00.94 /home/kats/zspace/gecko-fennec/mobile/android/build.mk:11: recipe for target 'package' failed
 0:00.94 make: *** [package] Error 2
 0:00.94 make: Leaving directory '/home/kats/zspace/gecko-fennec/obj-android-debug'

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 11 • 6 years ago

... and again. Does anybody have a workaround for this that I can apply when I hit it, so I don't have do a full clobber again?

	Kartikaya Gupta (email:kats@mozilla.staktrace.com) Reporter
	Comment 12 • 6 years ago

in this case it couldn't find dist/bin/chrome/en-US/local/en-US/pippki/certManager.dtd so I just deleted the dist/bin/chrome folder and ran "./mach build && ./mach package" and that seemed to work. So I guess that's a workaround. Hopefully it didn't produce a garbage APK.

	Nick Alexander :nalexander [he/him] (PTO until Montreallhands August 21, 2023)
	Comment 13 • 6 years ago

(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #11)
> ... and again. Does anybody have a workaround for this that I can apply when
> I hit it, so I don't have do a full clobber again?

You can be pretty confident that `rm -rf dist/bin && ./mach build && ./mach package` will be fine.  I think it used to be true that only `./mach package` populated `dist`, but that's no longer the case (especially after https://bugzilla.mozilla.org/show_bug.cgi?id=1255485 and similar tickets).