Closed
Bug 1122059
Opened 10 years ago
Closed 8 years ago
Use different filenames for each variant Android APK
Categories
(Release Engineering :: General, defect)
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: Sylvestre, Assigned: jlund)
References
Details
(Whiteboard: [get_apk.sh must be updated to accommodate this change])
Attachments
(1 file)
(deleted),
patch
|
Details | Diff | Splinter Review |
Files have the same name (fennec-38.0a1.multi.android-arm.apk):
http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android-api-11/
http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android-api-9/
This is a problem for release management:
* our tools does not support that (ok, that is not blocking but it would be harder to manage files with the same names)
* that introduces risks that we upload twice the same file
On the more general perspective, it introduces some potential confusions.
Reporter | ||
Updated•10 years ago
|
Summary: Please use different names for the APK → Please use different filenames for the APK
Comment 1•10 years ago
|
||
Can we use this bug to clean up releng's APK handling? The scripts do all kinds of special handling to find gecko-*.apk and robocop.apk. I want to upload lots of APK names and do different things with them in automation; for example, I want to upload both Proguarded and non-Proguarded APKs and use the non-Proguarded APKs in certain tests (JUnit 3 tests). I've run into tickets like Bug 919627 trying such things. And don't get me started on the nightmare of code that deploys robocop.apk to testing devices!
I propose that we include some {key: "file.apk"} in the tree configurations, either per mozharness job or in some other central location. I do not know if this is hard to achieve; I think for mozharness, it is not, but mozharness is only some small part of the issue.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=919627
Reporter | ||
Comment 2•10 years ago
|
||
(In reply to Nick Alexander :nalexander from comment #1)
> Can we use this bug to clean up releng's APK handling?
Sure, I was just explaining our (relman) expectations. Seems there are more ;)
Updated•10 years ago
|
Blocks: splitapk
Summary: Please use different filenames for the APK → Use different filenames for each variant Android APK
Updated•10 years ago
|
Component: Other → General Automation
QA Contact: pmoore → catlee
Reporter | ||
Comment 3•10 years ago
|
||
Chris, can you find someone to help with this bug?
We need a fix for this for the 37 cycle.
Flags: needinfo?(coop)
Comment 4•10 years ago
|
||
Assigning to :jlund to wrap-up the splitapk work.
Assignee: nobody → jlund
Flags: needinfo?(coop)
Assignee | ||
Comment 5•10 years ago
|
||
as mentioned here: https://bugzilla.mozilla.org/show_bug.cgi?id=1122509#c3 releng doesn't determine the apk name, we simply look for an apk file that was generated: http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#203
aiui, mobile team was experimenting with different package names last week here: "Bug 1120762 - Test, solidify, and document Google Play distribution approach for split APKs"
I'm not sure if they changed any filenames but either way, I'll keep this bug moving by chatting to names mentioned here: http://hg.mozilla.org/mozilla-central/annotate/eea6188b9b05/mobile/android/build.mk#l37
Reporter | ||
Comment 6•10 years ago
|
||
Richard, can you help with the new apk file name? Thanks
We need that to update our scripts.
Flags: needinfo?(rnewman)
Comment 7•10 years ago
|
||
Package names != filenames. I haven't touched the filenames, nor have I any idea how that works. Nick might.
Flags: needinfo?(rnewman) → needinfo?(nalexander)
Assignee | ||
Comment 8•10 years ago
|
||
(In reply to Richard Newman [:rnewman] from comment #7)
> Package names != filenames. I haven't touched the filenames, nor have I any
> idea how that works. Nick might.
sorry I meant you were changing package names and I wasn't sure if you experimented with changing filenames too.
I don't want to just try and hand responsibility off. That doesn't make sense since I started the apk split mess :)
Maybe if someone like nick (I also see mbrubeck, mfinkle, and blassey on build.mk contributor list) can get me up to speed with how we set filenames or enlighten me on any consequences of changing lines like http://hg.mozilla.org/mozilla-central/annotate/eea6188b9b05/mobile/android/build.mk#l37 from the in-tree compile side of things, that would be appreciated :D
nick, can you help me or point me to someone who may know more? thanks :)
Comment 9•10 years ago
|
||
(In reply to Jordan Lund (:jlund) from comment #8)
> (In reply to Richard Newman [:rnewman] from comment #7)
> > Package names != filenames. I haven't touched the filenames, nor have I any
> > idea how that works. Nick might.
>
> sorry I meant you were changing package names and I wasn't sure if you
> experimented with changing filenames too.
We have not. I have tried uploading multiple APK files and been sorely disappointed: Bug 919627, although that code looked to have changed when I last looked.
I have always been too scared to change APK file names since so much depends on "magic pattern matching": the build system has expectations, but the interaction between the build system and releng, and the updaters, also does. And things that scrape ftp.m.o. There's a long tail to chase down.
> I don't want to just try and hand responsibility off. That doesn't make
> sense since I started the apk split mess :)
>
> Maybe if someone like nick (I also see mbrubeck, mfinkle, and blassey on
> build.mk contributor list) can get me up to speed with how we set filenames
> or enlighten me on any consequences of changing lines like
> http://hg.mozilla.org/mozilla-central/annotate/eea6188b9b05/mobile/android/
> build.mk#l37 from the in-tree compile side of things, that would be
> appreciated :D
>
> nick, can you help me or point me to someone who may know more? thanks :)
All of the filename stuff is controlled by https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/package-name.mk. Package name here is historical and refers to the name of the build system output binary; it does /not/ refer to the "Android package name", which is a Java-style org.mozilla.* reverse-domain identifier.
I would talk to glandium.
Flags: needinfo?(nalexander)
Assignee | ||
Comment 10•10 years ago
|
||
pinged glandium over irc. adding ni here
Flags: needinfo?(mh+mozilla)
Comment 11•10 years ago
|
||
The build system actually has no expectations about the apk file name. Everything that does is on the releng side.
Flags: needinfo?(mh+mozilla)
Assignee | ||
Comment 12•10 years ago
|
||
I'm giving this a whirl here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ff9094b80b76
my assumption is that if a min sdk version is set, then it is a split build: either api-9-10 or api-11-* and we should append the apk name with 'api-MIN_SDK'
I'm hoping for the x86 android case, this will be ignored.
Although, this way seems fragile and bear in mind, this could be my first make patch ;)
Comment 13•10 years ago
|
||
Jordan,
I'm reluctantly showing I'm familiar with the build system here, I was going to recommend MOZ_PKG_SPECIAL in the relevant mozconfigs (like asan)
*however* it looks like android uses that as something else, specifically changing stuff for updater, as well as marking it as a low memory device.
(b2g uses it for other reasons too) http://mxr.mozilla.org/build/search?string=MOZ_PKG_SPECIAL&find=&findi=&filter=^[^\0]*%24&hitlimit=&tree=build
So your solution might indeed be the best bandaid afterall....
Assignee | ||
Comment 14•10 years ago
|
||
(In reply to Jordan Lund (:jlund) from comment #12)
> Created attachment 8573066 [details] [diff] [review]
> 150304_bug_1122059_use_diff_apk_name_for_split_m-c-2.patch
>
> I'm giving this a whirl here:
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=ff9094b80b76
>
ok, so that seemed to work as expected: apk generated fennec-39.0a1.en-US.android-arm-api-11.apk
but this breaks automation. looks like I need to change our regex here: http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#629
it looks for *arm.apk and we are appending to arm: arm-api-{9,11}.apk
I should be able to navigate to the appropriate fix from here :)
Updated•10 years ago
|
Whiteboard: [get_apk.sh must be updated to accommodate this change]
Updated•10 years ago
|
Assignee | ||
Comment 15•10 years ago
|
||
was pulled away for higher priorities. I had some time on Monday and today to look at this again. I'm currently trying to test this in staging.
Unfortunately, android builds still rely on buildbot factories (MozillaBuildFactory) and not mozharness so testing is harder.
At any rate, I'll report back on this tomorrow.
Assignee | ||
Comment 16•10 years ago
|
||
I've got a build working in staging but it is unconfirmed with testing. The issue here is controlling this rollout in a safe manner. I think we can do one of two things:
1) configure buildbot's android build factory to change the filename on one project branch and update mozharness tests to accept to accept both the old and new filenames everywhere. Then configure the new filename across trunk.
2) wait till end of quarter and address this when android builds are in mozharness + taskcluster: https://bugzil.la/1118394
the unknowns here is how this will affect our nightly, post upload automation, and release automation. e.g. Will we need to update bouncer links for grabbing nightlies? This is the area that I'm unfamiliar with and will be a pain dealing with while android builds are driven from buildbot. Unless this is blocking a release, my vote is to wait for mozharness+taskcluster builds and deal with this along side: https://bugzil.la/1122509
Sylvester: does that make sense? Rather than putting effort in tmp fixing this here, shall we wait for the multiple taskcluster efforts in Q2?
Flags: needinfo?(sledru)
Assignee | ||
Comment 17•10 years ago
|
||
fyi - I will be on PTO until May 13th.
the patch(es) I was playing with was: https://bugzilla.mozilla.org/attachment.cgi?id=8573066
and from buildbot-custom:
diff --git a/process/factory.py b/process/factory.py
index 0063b09..916cc9f 100644
--- a/process/factory.py
+++ b/process/factory.py
@@ -626,6 +626,12 @@ class MozillaBuildFactory(RequestSortingBuildFactory, MockMixin):
packageFilename = '*arm-armv6.apk'
elif 'android-x86' in self.complete_platform:
packageFilename = '*android-i386.apk'
+ elif 'android-api-9' in self.complete_platform:
+ # the arm.apk is to avoid unsigned/unaligned apks
+ packageFilename = '*arm-api-9.apk'
+ elif 'android-api-11' in self.complete_platform:
+ # the arm.apk is to avoid unsigned/unaligned apks
+ packageFilename = '*arm-api-11.apk'
elif 'android' in self.complete_platform:
# the arm.apk is to avoid unsigned/unaligned apks
packageFilename = '*arm.apk'
Reporter | ||
Comment 18•10 years ago
|
||
(In reply to Jordan Lund (:jlund) from comment #16)
> Sylvester: does that make sense? Rather than putting effort in tmp fixing
> this here, shall we wait for the multiple taskcluster efforts in Q2?
There is no rush here. If we are going to setup a clean and scalable solution, I would rather wait for it!
Flags: needinfo?(sledru)
Reporter | ||
Comment 19•9 years ago
|
||
Jordan, any update on this? We are discussing with a partner and this would help to avoid confusion.
Flags: needinfo?(jlund)
Assignee | ||
Comment 20•9 years ago
|
||
(In reply to Sylvestre Ledru [:sylvestre] from comment #19)
> Jordan, any update on this? We are discussing with a partner and this would
> help to avoid confusion.
android builds use taskcluster + mozharness now for m-c and m-a nightlies. We could probably change or at least experiment with changing the apk name pretty easily with those but it hasn't landed on m-b/m-r yet. Those repos are being blocked by and waiting on https://bugzil.la/1118794
iow we could proceed with switching m-c/m-a but that will result in a divergence in apk names in release branches for a while which I would assume would be confusing for release configs? Please needinfo me how you would like to proceed
Flags: needinfo?(jlund)
Reporter | ||
Comment 21•8 years ago
|
||
Now that we have much more automation here, this is no longer needed.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
Updated•7 years ago
|
Component: General Automation → General
Updated•6 years ago
|
You need to log in
before you can comment on or make changes to this bug.
Description
•