Closed Bug 1040319 Opened 11 years ago Closed 11 years ago

Ensure that Fennec builds from mozilla-esr31 have a buildID to allow for armv6/Android 2.2 users to update to mozilla-esr31 apks

Categories

(Release Engineering :: Release Automation, defect)

Product:
Release Engineering
Component:
Release Automation
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox32+ disabled, firefox-esr31 fixed, relnote-firefox 32+)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox32 + disabled
firefox-esr31 --- fixed
relnote-firefox --- 32+

People

(Reporter: lsblakk, Assigned: pmoore)

References

Details

Attachments

(7 files, 9 obsolete files)

bug1040319_tools_v2.patch
1.12 KB, patch
rail
: review+
rail
: feedback+
pmoore
: checked-in+
Details | Diff | Splinter Review
bug1040319tools.patch
10.46 KB, patch
pmoore
: review+
pmoore
: checked-in+
Details | Diff | Splinter Review
bug1040319_mozharness_v3.patch
2.92 KB, patch
rail
: review+
pmoore
: checked-in+
Details | Diff | Splinter Review
bug1040319_buildbot-configs_v4.patch
43.35 KB, patch
rail
: review+
pmoore
: checked-in+
Details | Diff | Splinter Review
bug1040319_esr31_branch_v2.patch
1.43 KB, patch
pmoore
: review+
lmandel
: approval-mozilla-esr31+
Details | Diff | Splinter Review
bug1040319_esr31_branch_beta_branding_v1.patch
9.49 KB, patch
Details | Diff | Splinter Review
bug1040319_build_script.sh
21.79 KB, text/plain
rail
: review+
Details
This is the tracking bug for the results of today's meeting about EOL strategy for Android 2.2 & armv6. "Effective next week we will take uplifts up to FF32 (beta) that will turn off armv6 and block installation of Firefox for Android to Android 2.2 users. This will take effect on Nightly tomorrow and then Aurora/Beta after uplift. For releases over the next 6 months there will be builds of armv6 (supporting Android 2.2) off the mozilla-esr31 branch, but this is not an ESR. Matt Wobensmith who does the security QA for Firefox ESR will have no additional work specific to mobile releases from this branch. Mobile QA will maintain a comparable level of effort on releases, with two builds (x86, armv7) from mozilla-release and then one (armv6) from mozilla-esr31. To get ready for the first of these releases, in 6 weeks time when we ship FF32, RelEng has an action on them to confirm that they can build from the two repos with two separate requests for builds and get the buildID set correctly on them to ensure updates will be offered to the Android 2.2 users from the Play Store. QA will also have an action item to confirm this works and that users do not need to uninstall/reinstall to be on the security-fix-only builds." To resolve this bug fixed please confirm the buildID generation still works for the three apks as it does now when using a single repo. QA needs to sign off that updates from FF31 GA to FF31.1 from mozilla-esr31 works. Finally, flagging for relnote-firefox? as this will be one of the channels through which we can announce the de-support and that security-fix-only builds will end on January 6th.
armv6 needs to be the lowest version code. I was brainstorming that if ESR31 had code to subtract, say, 1 week from the build date, we could avoid tricky go-to-build timing issues. Otherwise, if we kept the subtract-one-{hour?} from the build date, we'd have to make sure the esr build started in the same hour or before the release build.
[13:33] <aki> coop: http://mxr.mozilla.org/mozilla-beta/source/mobile/android/base/Makefile.in#17 [13:33] <aki> specifically http://mxr.mozilla.org/mozilla-beta/source/mobile/android/base/Makefile.in#22 [13:33] <coop> aki: thanks [13:33] <aki> should be a one liner [13:35] <aki> make sure he knows it doesn't have to be a valid datetime string; subtracting 7*24 (or whatever units it's in) is valid, even if that means it goes to day 97 of the previous month [13:35] <aki> just numbers that need to be greater than or less than [13:36] <catlee> I thought I put that in the bug? [13:36] <aki> dunno, i haven't read the bug :) [13:36] <aki> https://bugzilla.mozilla.org/show_bug.cgi?id=1040319
http://hg.mozilla.org/releases/mozilla-esr31/file/1e303589a899/mobile/android/base/Makefile.in#l22 since I think mxr's links aren't permanent (adding or subtracting lines at the beginning of the file will make the line link point to the wrong line).
(In reply to Aki Sasaki [:aki] from comment #2) > [13:35] <aki> make sure he knows it doesn't have to be a valid datetime > string; subtracting 7*24 (or whatever units it's in) is valid, even if that > means it goes to day 97 of the previous month > [13:35] <aki> just numbers that need to be greater than or less than I thought more about this, and this might suck if build day is the last of the month, and the ESR gets built a day later, on the first of the month. =\ Trying to figure out a way other than 1) convert to epoch time 2) subtract the correct # of seconds 3) re-convert to datetime of the appropriate format, but that may be the cleanest solution.
Pete - Ben tells me that you've picked up this work. Do you have a plan for testing this on beta before we hit release? What is your time line for completing this work?
Flags: needinfo?(pmoore)
Hi Lawrence, That's right - I'm working on this over in bug 1042128 which I guess we can mark as a duplicate (or this a duplicate of that). In any case, I can briefly summarize my proposal here: The versionCode in the apk meta data (which is derived from the buildid) needs to have a lower value for armv6 than the other architectures. It is treated as a plain integer. My proposal is to change the mechanism to generate the versionCode from a starting point of e.g. 2008080001, and then bump it by 1 with each release. This guarantees that: a) the version number is higher than the previous releases for armv6 releases b) it continues to increase c) it will always take less precedence than the versionCode of the apk's for the armv7 and x86 apks. If you'd like to review the full content of bug 1042128 (which gives a bit more detail) and let me know your feedback on this proposal, I'd be very happy to hear it. I asked Nick, Ben, Coop and Catlee also for their feedback, and I think we're all aligned currently that this proposal should work - but maybe you spot something we haven't. :-) Many thanks, Pete
Flags: needinfo?(pmoore)
Blocks: 1042845
Assignee: nobody → pmoore
Depends on: 1048971
For transparency, attaching my work so far, as it is end-of-day now and maybe someone wants to see it before I am back at work in the morning...
Attached patch bug1040319_esr31_branch_v1.patch (obsolete) — — Splinter Review
Attached patch bug1040319_tools_v1.patch (obsolete) — — Splinter Review
Attachment #8475998 - Flags: feedback?(rail)
Attachment #8475999 - Flags: feedback?(rail)
Attachment #8476000 - Flags: feedback?(rail)
I guess I will also need a similar patch for betas? Would that be release-fennec-mozilla-beta.py.template? I guess both betas and releases will get made off the esr31 branch(?) Hmmmm... What do you think, Rail? Not quite sure how that works.
Flags: needinfo?(rail)
Attachment #8475999 - Flags: feedback?(rail) → feedback+
Comment on attachment 8475998 [details] [diff] [review] bug1040319_buildbot-configs_v1.patch Review of attachment 8475998 [details] [diff] [review]: ----------------------------------------------------------------- Since this "hack will be going away when stop support for armv6, I'm OK to have a function in a config file to avoid pollution of libraries and scripts. ::: mozilla/release-firefox-mozilla-esr31.py.template @@ +42,5 @@ > releaseConfig['nextMilestone'] = releaseConfig['nextAppVersion'] > + > +def bumpIntegerInFile(previousContents): > + print previousContents[0] > + return str(int(previousContents[0]) + 1) Why you pass a tuple to this function? Wouldn't it be safer to pass a string to avoid IndexErrors?
Attachment #8475998 - Flags: feedback?(rail) → feedback+
Comment on attachment 8476000 [details] [diff] [review] bug1040319_tools_v1.patch Review of attachment 8476000 [details] [diff] [review]: ----------------------------------------------------------------- ::: lib/python/build/versions.py @@ +26,5 @@ > + # If version is a function, it will do the bump. > + # It takes the old contents as its input to generate > + # the new content. > + if callable(version): > + return version((contents,)) See my previous comment about passing a string instead of a tuple. Also, calling "version" doesn't sound right to me... Maybe rename the variable or instead of using bumpFile() call the callable from scripts/release/tag-release.py directly?
Attachment #8476000 - Flags: feedback?(rail) → feedback+
(In reply to Pete Moore [:pete][:pmoore] from comment #12) > I guess I will also need a similar patch for betas? Would that be > release-fennec-mozilla-beta.py.template? > > I guess both betas and releases will get made off the esr31 branch(?) > Hmmmm... What do you think, Rail? Not quite sure how that works. From what I saw you don't need any changes in other config files.
Flags: needinfo?(rail)
(In reply to Rail Aliiev [:rail] from comment #13) > Comment on attachment 8475998 [details] [diff] [review] > bug1040319_buildbot-configs_v1.patch > > Review of attachment 8475998 [details] [diff] [review]: > ----------------------------------------------------------------- > > Since this "hack" will be going away when stop support for armv6, I'm OK to > have a function in a config file to avoid pollution of libraries and scripts. Thanks Rail. This was my thinking too. Then as much of the changes as possible are localised to the esr31 branch, which will die, when it does. This was my thinking too. > ::: mozilla/release-firefox-mozilla-esr31.py.template > @@ +42,5 @@ > > releaseConfig['nextMilestone'] = releaseConfig['nextAppVersion'] > > + > > +def bumpIntegerInFile(previousContents): > > + print previousContents[0] > > + return str(int(previousContents[0]) + 1) > > Why you pass a tuple to this function? Wouldn't it be safer to pass a string > to avoid IndexErrors? Whoops. That indeed is nonsense - left over mess from debugging/experimenting. Will fix! Sorry, I rushed it out a bit last night as I had to rush off for an appointment, and wanted to just get a sanity check that you were happy with the approach - but yes, very silly wrapping in a tuple! :D
Hey Rail, Since we are releasing fennec from esr 31 branch, which we've not done before, I realised we need to create new configs for this. When setting up my staging environment with a local ship it and release runner instance, I discovered that the first time the staging release runs, release runner seems to require the existence of the previously generated release config file (i.e. the file that is generated from the .template file). For this reason, I have also included staging_release-fennec-mozilla-esr31.py in this patch, even though it is generated from staging_release-fennec-mozilla-esr31.py.template - since I believe release runner requires the file to exist, before it gets generated. If this is not the case, and I'm doing something wrong, let me know! I've applied your previous suggestions in these patches too. Again, this is just a feedback request, rather than a full review, since I wanted to show how I was progressing. For example, my buildbot configs reference mozharness single and multi locale configs that I have not generated yet. I also enabled nightly builds for armv6 as I am guessing this will be needed - although I believe I'll need to make further changes to buildbot configs to get them enabled just on esr31 branch - however, if you think we shouldn't need nightly builds for esr31, and dep builds only are required, let me know! This is pretty much my first work in buildbot configs, so I wanted some feedback early - I appreciate there may still be some further changes required. Thanks for your patience and attention! :) Pete
Attachment #8475998 - Attachment is obsolete: true
Attachment #8478646 - Flags: feedback?(rail)
Attachment #8476000 - Attachment is obsolete: true
Attachment #8478647 - Flags: feedback?(rail)
Attachment #8478647 - Flags: feedback?(rail) → feedback+
Comment on attachment 8478646 [details] [diff] [review] bug1040319_buildbot-configs_v2.patch Review of attachment 8478646 [details] [diff] [review]: ----------------------------------------------------------------- I overall it looks good, just needs some corrections. ::: mozilla/release-fennec-mozilla-esr31.py.template @@ +75,5 @@ > + 'build/mozharness': 'production', > +} > + > +# Platform configuration > +releaseConfig['enUSPlatforms'] = ('android', 'android-x86') ('android-armv6',) I believe @@ +82,5 @@ > +releaseConfig['talosTestPlatforms'] = () > +releaseConfig['enableUnittests'] = False > + > +# L10n configuration > +releaseConfig['l10nPlatforms'] = ('android',) android-armv6 here, I think? @@ +124,5 @@ > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { > + 'android': [ android-armv6? ::: mozilla/staging_release-fennec-mozilla-esr31.py @@ +1,1 @@ > +# ATTENTION: Just "touch staging_release-fennec-mozilla-esr31.py" to make release-runner happy. In this case the automatic patch generated by release-runner will look readable. ::: mozilla/staging_release-fennec-mozilla-esr31.py.template @@ +84,5 @@ > +releaseConfig['talosTestPlatforms'] = () > +releaseConfig['enableUnittests'] = False > + > +# L10n configuration > +releaseConfig['l10nPlatforms'] = ('android',) android-armv6? @@ +126,5 @@ > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { > + 'android': [ android-armv6?
Attachment #8478646 - Flags: feedback?(rail) → feedback-
Blocks: 1058888
Attached patch bug1040319_mozharness_v1.patch (obsolete) — — Splinter Review
Attachment #8479346 - Flags: review?(bhearsum)
Comment on attachment 8478646 [details] [diff] [review] bug1040319_buildbot-configs_v2.patch Review of attachment 8478646 [details] [diff] [review]: ----------------------------------------------------------------- ::: mozilla/release-fennec-mozilla-esr31.py.template @@ +75,5 @@ > + 'build/mozharness': 'production', > +} > + > +# Platform configuration > +releaseConfig['enUSPlatforms'] = ('android', 'android-x86') Quite right - fixed. @@ +82,5 @@ > +releaseConfig['talosTestPlatforms'] = () > +releaseConfig['enableUnittests'] = False > + > +# L10n configuration > +releaseConfig['l10nPlatforms'] = ('android',) Indeed - fixed. @@ +124,5 @@ > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { > + 'android': [ I don't think so here - I checked all other release configs that had the single_locale_options, and all of them specified 'android' and none of them specified e.g. 'android-x86' also when android x86 builds were available for that branch - so I think this may not depend on the chipset architecture(?) That said, I am only basing this on what I see in other configs, I'm not sure how this config is used. ::: mozilla/staging_release-fennec-mozilla-esr31.py @@ +1,1 @@ > +# ATTENTION: This is what I originally tried to do, but I still got errors. However, I'll try again and report back what the error was. ::: mozilla/staging_release-fennec-mozilla-esr31.py.template @@ +84,5 @@ > +releaseConfig['talosTestPlatforms'] = () > +releaseConfig['enableUnittests'] = False > + > +# L10n configuration > +releaseConfig['l10nPlatforms'] = ('android',) Thanks - fixed. @@ +126,5 @@ > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { > + 'android': [ Same as above - I don't think so here.
Hey Rail, I realise after publishing that the comment above looks weird (my responses are to the code, not to your comments) - if you go to https://bugzilla.mozilla.org/page.cgi?id=splinter.html&bug=1040319&attachment=8478646 and click "All Files" you will see that my comments in response to your comments inline. Thanks! Pete
Flags: needinfo?(rail)
> > @@ +124,5 @@ > > > +releaseConfig['enableUpdatePackaging'] = False > > > +releaseConfig['balrog_api_root'] = None > > > + > > > +releaseConfig['single_locale_options'] = { > > > + 'android': [ > > > > android-armv6? > > > I don't think so here - I checked all other release configs that had the > single_locale_options, and all of them specified 'android' and none of them > specified e.g. 'android-x86' also when android x86 builds were available for > that branch - so I think this may not depend on the chipset architecture(?) > > That said, I am only basing this on what I see in other configs, I'm not > sure how this config is used. OK, I checked the code - and you are quite right. I see here we reference releaseConfig['single_locale_options'][platform] if 'android' is in the platform name: https://hg.mozilla.org/build/buildbotcustom/file/1dec08be4399/process/release.py#l810 and platform comes from the list in releaseConfig['enUSPlatforms']: https://hg.mozilla.org/build/buildbotcustom/file/1dec08be4399/process/release.py#l585 which indeed contains 'android-armv6' in our config. So I'll change this too! I should no better than to question! :D Sorry about that. Pete
Flags: needinfo?(rail)
s/no better/know better/ *eyeroll*
Feedback incorporated!
Attachment #8478646 - Attachment is obsolete: true
Attachment #8479798 - Flags: review?(rail)
Comment on attachment 8479346 [details] [diff] [review] bug1040319_mozharness_v1.patch Review of attachment 8479346 [details] [diff] [review]: ----------------------------------------------------------------- ::: configs/single_locale/release_mozilla-esr_android-armv6.py @@ +16,5 @@ > + "buildbot_json_path": "buildprops.json", > + "purge_minsize": 10, > + "force_clobber": True, > + "clobberer_url": "http://clobberer.pvt.build.mozilla.org/index.php", > + "locales_file": "buildbot-configs/mozilla/l10n-changesets_mozilla-esr31", We use JSON files like https://github.com/mozilla/build-buildbot-configs/blob/master/mozilla/l10n-changesets_mobile-release.json for these releases, I think this name needs to be updated. r=me with that fixed.
Attachment #8479346 - Flags: review?(bhearsum) → review+
Attached patch bug1040319_mozharness_v2.patch (obsolete) — — Splinter Review
Nice spot! Thanks Ben! :)
Attachment #8479346 - Attachment is obsolete: true
Attachment #8479893 - Flags: review?(bhearsum)
Comment on attachment 8479798 [details] [diff] [review] bug1040319_buildbot-configs_v3.patch Review of attachment 8479798 [details] [diff] [review]: ----------------------------------------------------------------- Almost there. See my comments below. ::: mozilla/release-fennec-mozilla-esr31.py.template @@ +124,5 @@ > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { > + 'android-armv6': [ I looked where this config variable is used. http://hg.mozilla.org/build/buildbotcustom/file/1dec08be4399/process/release.py#l810 uses it to define single locale extra params for the platforms listed in releaseConfig['l10nPlatforms']. In your case this is ("android-armv6",). We don't support single locales for armv6, this is why our regular configs redefine releaseConfig['l10nPlatforms'] as ("android",) instead of reusing releaseConfig['enUSPlatforms']. See also http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/31.0-candidates/build1/android/ vs http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/31.0-candidates/build1/android-armv6/ In your case releaseConfig['l10nPlatforms'] should be set to () (and empty tuple). It won't affect the multilocale build which is used as default APK for Google Play. Additionally you can delete releaseConfig['single_locale_options'] all together since it's irrelevant for armv6. ::: mozilla/staging_release-fennec-mozilla-esr31.py @@ +84,5 @@ > +releaseConfig['talosTestPlatforms'] = () > +releaseConfig['enableUnittests'] = False > + > +# L10n configuration > +releaseConfig['l10nPlatforms'] = releaseConfig['enUSPlatforms'] The same here. Needs to be set to () @@ +125,5 @@ > +releaseConfig['disablePushToMirrors'] = True > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { Kill this variable. ::: mozilla/staging_release-fennec-mozilla-esr31.py.template @@ +84,5 @@ > +releaseConfig['talosTestPlatforms'] = () > +releaseConfig['enableUnittests'] = False > + > +# L10n configuration > +releaseConfig['l10nPlatforms'] = releaseConfig['enUSPlatforms'] () @@ +125,5 @@ > +releaseConfig['disablePushToMirrors'] = True > +releaseConfig['enableUpdatePackaging'] = False > +releaseConfig['balrog_api_root'] = None > + > +releaseConfig['single_locale_options'] = { 9dd
Attachment #8479798 - Flags: review?(rail) → review-
Attachment #8479893 - Flags: review?(bhearsum) → review+
pmoore: you need a release-fennec-mozilla-esr31.py file in addition to release-fennec-mozilla-esr31.py.template, it is missing in your patches and checkconfig fails on my build dev-master
Also, these patches don't checkconfig on a dev-master. You need to add mozilla-esr31 to the mobile_release_branches in master/master_config.json to enable the platform. Which means there are tools patches as well in production_masters.json for the associated masters that run mobile release builds.
Attached patch bug1040319bbconfigs.patch (obsolete) — — Splinter Review
Pete: these are the patches I applied to my dev-master to get the builders for ESR32 for Android to appear. Also, I had to add links to the new file [kmoir@dev-master1.srv.releng.scl3.mozilla.com master]$ history | grep ln 1068 ln -s /builds/buildbot/kmoir/build3/buildbot-configs/mozilla/staging_release-fennec-mozilla-esr31.py staging_release-fennec-mozilla-esr31.py 1073 ln -s /builds/buildbot/kmoir/build3/buildbot-configs/mozilla/release-fennec-mozilla-esr31.py release-fennec-mozilla-esr31.py 1131 history | grep ln [kmoir@dev-master1.srv.releng.scl3.mozilla.com master]$ pwd /builds/buildbot/kmoir/build3/master and "mobile_release_branches": ["mozilla-release", "mozilla-beta", "mozilla-esr31" ], in master_config.json.
Sorry previous comment should have said esr31, not esr32. Also, here is the master with the jobs http://dev-master1.srv.releng.scl3.mozilla.com:8036/builders Next step is apply these patches to your master, attach slaves, and try a staging release. I'll attach patches for the tools repo to include this new branch in production_masters.json
patch to add mozilla-esr31 branch to masters that build mobile releases
Attachment #8480147 - Flags: review?(pmoore)
Comment on attachment 8480147 [details] [diff] [review] bug1040319tools.patch Review of attachment 8480147 [details] [diff] [review]: ----------------------------------------------------------------- lgtm, thanks Kim!
Attachment #8480147 - Flags: review?(pmoore) → review+
(In reply to Kim Moir [:kmoir] from comment #31) > Created attachment 8480135 [details] [diff] [review] > bug1040319bbconfigs.patch > > Pete: these are the patches I applied to my dev-master to get the builders > for ESR32 for Android to appear. > > Also, I had to add links to the new file > [kmoir@dev-master1.srv.releng.scl3.mozilla.com master]$ history | grep ln > 1068 ln -s > /builds/buildbot/kmoir/build3/buildbot-configs/mozilla/staging_release- > fennec-mozilla-esr31.py staging_release-fennec-mozilla-esr31.py > 1073 ln -s > /builds/buildbot/kmoir/build3/buildbot-configs/mozilla/release-fennec- > mozilla-esr31.py release-fennec-mozilla-esr31.py > 1131 history | grep ln > [kmoir@dev-master1.srv.releng.scl3.mozilla.com master]$ pwd > /builds/buildbot/kmoir/build3/master Thanks Kim. These already existed for me, but I think they are probably created by https://hg.mozilla.org/build/buildbot-configs/file/eedf7a8d2651/setup-master.py#l71 during the buildbot installation (but with an existing installation, the links need to be manually created like you did, as I guess setup-master.py only runs during initial installation). > and "mobile_release_branches": ["mozilla-release", "mozilla-beta", > "mozilla-esr31" ], in master_config.json. Awesome - I've done this too now. Thanks again Kim!
Comment on attachment 8480135 [details] [diff] [review] bug1040319bbconfigs.patch Review of attachment 8480135 [details] [diff] [review]: ----------------------------------------------------------------- ::: mozilla/release-fennec-mozilla-esr31.py @@ +39,5 @@ > +# Repository configuration, for tagging > +releaseConfig['sourceRepositories'] = { > + 'mobile': { > + 'name': 'mozilla-esr31', > + 'clonePath' : 'releases/mozilla-esr31', I only found clonePath in staging configs. But I think this file is generated anyway, so it probably does no harm, as the file should get overwritten.
Depends on: 1059890
Release noted in Firefox 32 as "Android 2.2 and ARMv6 processor chipset no longer supported"
relnote-firefox: ? → 32+
Attachment #8475999 - Flags: review?(mshal)
Comment on attachment 8475999 [details] [diff] [review] bug1040319_esr31_branch_v1.patch Review of attachment 8475999 [details] [diff] [review]: ----------------------------------------------------------------- Hey Catlee, If you r+ this, please can you also land for me, as I think I don't have rights to land in-tree. I've tested building an apk, and that the version code is correctly generated. I have not tested downloading from the play store with multiple different handsets supporting different architectures (armv6/armv7/x86), to confirm that the correct apk is offered in each case, however Kevin Brosnan will be doing this in QA team, I believe. Any questions, let me know. When you are happy with the patch, I have permission from Lawrence Mandel to run 31.1b1 build 1 with these changes. Thanks, Pete
Attachment #8475999 - Flags: review?(mshal) → review?(catlee)
Attachment #8475999 - Flags: review?(catlee) → review+
Comment on attachment 8475999 [details] [diff] [review] bug1040319_esr31_branch_v1.patch [Approval Request Comment] If this is not a sec:{high,crit} bug, please state case for ESR consideration: User impact if declined: No armv6 builds of fennec Fix Landed on Version: not yet pushed Risk to taking this patch (and alternatives if risky): none - currently no fennec builds on esr31 branch so worst case, fennec builds fail and nothing is lost String or UUID changes made by this patch: no l10n changes with this patch See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.
Attachment #8475999 - Flags: approval-mozilla-esr31?
Comment on attachment 8475999 [details] [diff] [review] bug1040319_esr31_branch_v1.patch Approved for ESR31.
Attachment #8475999 - Flags: approval-mozilla-esr31? → approval-mozilla-esr31+
So I have some good news! Staging runs ran through to completion: http://dev-stage01.srv.releng.scl3.mozilla.com/pub/mozilla.org/mobile/candidates/31.1.0esr-candidates/build12/android-armv6/multi/ Please note these are *staging* builds signed with staging keys. I'm now going to be landing all the changes to our production repos, and creating the official 31.1.0esr release with the correct signing keys and correct locales, etc. Another useful validation is that the manifest indeed seems to be getting updated correctly, e.g.: pmoore@Elisandra:~ $ ssh cltbld@dev-linux64-ec2-pmoore.dev.releng.use1.mozilla.com head -10 /builds/slave/rel-m-esr31-and-a6_bld-0000000/build/obj-firefox/mobile/android/base/AndroidManifest.xml <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="org.mozilla.firefox" android:installLocation="auto" android:versionCode="2014080006" <-- the new versionCode android:versionName="31.1.0" android:sharedUserId="org.mozilla.firefox.sharedID" > <uses-sdk android:minSdkVersion="8" android:targetSdkVersion="17"/> pmoore@Elisandra:~ $ and this perfectly corresponds to it being the 6th iteration of the build, i.e. it was generated by: http://dev-master1.srv.releng.scl3.mozilla.com:8444/builders/release-mozilla-esr31-android-armv6_build/builds/6 Therefore it appears to be correctly bumping the versionCode by one with each release made. This can also be seen here: https://hg.mozilla.org/users/pmoore_mozilla.com/mozilla-esr31/rev/cf91eff58905 So both parts are working correctly: 1) Counter is getting incremented with each release 2) Release process is bumping the file in source control, so the increase will not be "forgotten" I'll update this bug with the latest patches for all the repos. It may take a couple of hours to prepare this, as I will need to strip out all the staging config changes in the process, where they were necessary. Please note this is a multilocale build, there are not individual locales. It is my understanding that this is what the requirements are.
Updated version based on what was successfully used in staging.
Attachment #8479893 - Attachment is obsolete: true
Attachment #8484342 - Flags: review?(bhearsum)
Comment on attachment 8480147 [details] [diff] [review] bug1040319tools.patch Marking as obsolete as it is included in a later aggregated patch in this bug.
Attachment #8480147 - Attachment is obsolete: true
Comment on attachment 8480135 [details] [diff] [review] bug1040319bbconfigs.patch Marking as obsolete since it is included in a later aggregated patch in this bug.
Attachment #8480135 - Attachment is obsolete: true
This is the "normalised" patch I get after removing staging-specific parts in files that are also used in prod - i.e. a cleaned up version of the delta I had in my staging environment, stripped of the hacks needed to get staging working.
Attachment #8479798 - Attachment is obsolete: true
Attachment #8484406 - Flags: review?(bhearsum)
Comment on attachment 8484342 [details] [diff] [review] bug1040319_mozharness_v3.patch Ben/Rail - adding you both so whoever gets there first can take it. Thanks guys! Pete
Attachment #8484342 - Flags: review?(rail)
Comment on attachment 8484406 [details] [diff] [review] bug1040319_buildbot-configs_v4.patch And this one too...
Attachment #8484406 - Flags: review?(rail)
Comment on attachment 8475999 [details] [diff] [review] bug1040319_esr31_branch_v1.patch I'm not able to land this change - can someone land this for me? pmoore@Elisandra:/Volumes/TOSHIBA EXT/mozilla-esr31 $ hg out *** failed to import extension hggit: No module named dulwich.errors comparing with ssh://hg.mozilla.org/releases/mozilla-esr31/ searching for changes changeset: 200301:6ae5e6c8ede9 tag: tip user: Peter Moore <pmoore@mozilla.com> date: Thu Sep 04 23:12:46 2014 +0200 summary: Bug 1040319 - Ensure that Fennec builds from mozilla-esr31 have a buildID to allow for armv6/Android 2.2 users to update to mozilla-esr31 apks,r=catlee,a=lmandel pmoore@Elisandra:/Volumes/TOSHIBA EXT/mozilla-esr31 $ hg push *** failed to import extension hggit: No module named dulwich.errors pushing to ssh://hg.mozilla.org/releases/mozilla-esr31/ searching for changes remote: abort: could not lock repository /repo/hg/mozilla/releases/mozilla-esr31/: Permission denied abort: unexpected response: empty string pmoore@Elisandra:/Volumes/TOSHIBA EXT/mozilla-esr31 $
Flags: needinfo?(lmandel)
Same patch as before, but now with commit headers etc (generated with hg export) so that somebody can land it for me (I don't have permission). Thanks, Pete
Attachment #8475999 - Attachment is obsolete: true
Attachment #8484509 - Flags: review+
Attachment #8484509 - Flags: approval-mozilla-esr31?
Comment on attachment 8478647 [details] [diff] [review] bug1040319_tools_v2.patch Review of attachment 8478647 [details] [diff] [review]: ----------------------------------------------------------------- This was previously a feedback+ so adding a r? for landing it. Thanks! Pete
Attachment #8478647 - Flags: review?(rail)
Please note: as soon as all 4 patches land and buildbot-configs and mozharness changes are merged to production branches, the official Fennec 31.1.0esr build 1 can be triggered in ship it (reconfig will be done automatically by release runner).
Attachment #8478647 - Flags: review?(rail) → review+
Comment on attachment 8484342 [details] [diff] [review] bug1040319_mozharness_v3.patch Review of attachment 8484342 [details] [diff] [review]: ----------------------------------------------------------------- lgtm
Attachment #8484342 - Flags: review?(rail) → review+
Comment on attachment 8484509 [details] [diff] [review] bug1040319_esr31_branch_v2.patch pmoore added Hg headers as he wasn't able to land himself. Adding approval back to the patch.
Attachment #8484509 - Flags: approval-mozilla-esr31? → approval-mozilla-esr31+
Flags: needinfo?(lmandel)
Attachment #8484342 - Flags: review?(bhearsum)
Comment on attachment 8484406 [details] [diff] [review] bug1040319_buildbot-configs_v4.patch Review of attachment 8484406 [details] [diff] [review]: ----------------------------------------------------------------- ::: mozilla/release-fennec-mozilla-esr31.py @@ +30,5 @@ > +# Current version info > +releaseConfig['version'] = '31.1.0esr' > +releaseConfig['appVersion'] = '31.1.0' > +releaseConfig['milestone'] = releaseConfig['appVersion'] > +releaseConfig['buildNumber'] = 12 A nit. Can you make it 1 instead of 12, just to have better incremental diffs.
Attachment #8484406 - Flags: review?(rail) → review+
Thanks Rail for all your reviews! =) Thanks Ryan for landing! Will wait for jenkins to run tests for buildbot-configs changes, and then I will trigger Fennec 31.1.0esr build 1 on production ship it. Please note, mobile tests have not yet been enabled - I'll create a bug for this so we can set it up asap. Pete
Depends on: 1063420
See Also: → 1063438
See Also: → 1063442
Blocks: 1048971
No longer depends on: 1048971
(In reply to Pete Moore [:pete][:pmoore] from comment #45) > Comment on attachment 8480147 [details] [diff] [review] > bug1040319tools.patch > > Marking as obsolete as it is included in a later aggregated patch in this > bug. Whoops - no it wasn't. Setting back.
Attachment #8480147 - Attachment is obsolete: false
From :kbrosnan: I ran our smoke tests against the builds, there were some build problems and test failures https://moztrap.mozilla.org/runtests/run/5327/env/26/ First of this is a release Firefox for Android build (org.mozilla.firefox). We cannot ship it to Firefox Beta (org.mozilla.firefox_beta) users as we had planned. * No release notes, www.mozilla.org/en-US/mobile/31.1.0/releasenotes * Droid Incredible (2.2) YouTube app seems to be out of date, thus clicking on a m.youtube.com link presents an error message from the Google Youtube app. I do not believe this to be a Firefox issue. No updates for YT available in the Play Store. Using an Samsung SCH-R720 running Android 2.3.4 is able to launch the YouTube app correctly. * Allowing popups fails to work need to check this behavior vs 31 release Kevin Brosnan
Hi Rail, Thanks for your tip about reversing http://hg.mozilla.org/releases/mozilla-release/rev/a3cc3e46b571 to force the build system to use: http://hg.mozilla.org/releases/mozilla-release/file/a3cc3e46b571/mobile/android/branding/beta/configure.sh - this is what I have done in this patch. Many thanks, Pete
Attachment #8490842 - Flags: review?(rail)
The patch itself looks good to me, I have a couple of concerns regarding the procedure: * If the change is a one off, we could probably build manually, without landing the patch. This is a bit error-prone. :/ * The automation will be generating another build under 31.x.x-candidates directory, while the binaries are more like 31.x.xbx-candidates. Maybe we should move the files somewhere else manually if we decide to use automation to build this. * If we decide to use this patch and our release automation to generate the builds, I'd land the patch on a separate branch (not default), so we don't even care about backing it out. Branching off of MOBILE3110esr_2014090505_RELBRANCH sounds like a candidate. What is the plan here?
Flags: needinfo?(pmoore)
(In reply to Rail Aliiev [:rail] from comment #66) > The patch itself looks good to me, I have a couple of concerns regarding the > procedure: > > * If the change is a one off, we could probably build manually, without > landing the patch. This is a bit error-prone. :/ > > * The automation will be generating another build under 31.x.x-candidates > directory, while the binaries are more like 31.x.xbx-candidates. Maybe we > should move the files somewhere else manually if we decide to use automation > to build this. Primarily for this reason, I'm pretty in favour of building by hand. The builds are not terribly difficult, and it's easier not to clutter up FTP and the repository with stuff for a one-off IMO. There's no reason for these builds to ever be on FTP as far as I can tell - we just need them for a test in the Google Play store. > > * If we decide to use this patch and our release automation to generate the > builds, I'd land the patch on a separate branch (not default), so we don't > even care about backing it out. Branching off of > MOBILE3110esr_2014090505_RELBRANCH sounds like a candidate. +1, if we go with automation.
The beta build is a one off.
Comment on attachment 8490842 [details] [diff] [review] bug1040319_esr31_branch_beta_branding_v1.patch We agreed on IRC to proceed with a manual one off build to avoid cluttering repo/ftp.
Attachment #8490842 - Flags: review?(rail)
Flags: needinfo?(pmoore)
Running a manual build on dev-linux64-ec2-pmoore.dev.releng.use1.mozilla.com
The build has worked, quite miraculously! I'm just at the closing stages now, I'll attach the script I used when I've got to the end... I generated the script with the help of my good friend sed, and it pretty much worked off the bat (although took an age to actually build): https://hg.mozilla.org/build/braindump/raw-file/fc6ff19493c1/buildbot-related/scrape_buildbot_logs_for_commands.sed Will update shortly with details about location of new apk.
Uploaded to: /home/pmoore/fennec-31.1.0esr.en-US.android-arm-armv6.apk on people.mozilla.org
Flags: needinfo?(kbrosnan)
Attached file bug1040319_build_script.sh (obsolete) —
This script was generated with: curl -L 'http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/31.1.0esr-candidates/build1/logs/release-mozilla-esr31-android-armv6_build-bm85-build1-build0.txt.gz' 2>/dev/null | gunzip | ./scrape_buildbot_logs_for_commands.sed > all_commands.sh where scrape_buildbot_logs_for_commands.sed is: https://hg.mozilla.org/build/braindump/raw-file/fc6ff19493c1/buildbot-related/scrape_buildbot_logs_for_commands.sed And then any tweaks I needed to make (steps to skip, for example, or creating the initial directories) I have added with comments, so it is straightforward to see the deviations I added to the original run script. Essentially it skips the upload steps, and after the package has been made (make package), all subsequent steps are skipped. Additionally, after esr31 repo has been checked out, the script patches the repo with the fix from comment 65. Lastly, bhearsum showed me how to manually generate a signing token (many thanks!) so that we could get the signing to work. I ran chmod 644 on the apk on people.mozilla.org in my home directory so that it can be downloaded by anyone with access to people, to avoid confusing matters with firefox beta apks being uploaded to ftp.
Comment on attachment 8492346 [details] bug1040319_build_script.sh >python /builds/slave/rel-m-esr31-and-a6_bld-0000000/tools/buildfarm/utils/retry.py -s 1 -r 5 -t 1260 bash -c 'if [ -f "mobile/android/config/mozconfigs/android-armv6/release" ]; then echo Using in-tree mozconfig; cp mobile/android/config/mozconfigs/android-armv6/release .mozconfig; else echo Downloading mozconfig; wget -O .mozconfig https://hg.mozilla.org/build/buildbot-configs/raw-file/FENNEC_31_1_0esr_RELEASE/mozilla2/android-armv6/mozilla-esr31/release/mozconfig; fi' >cd /builds/slave/rel-m-esr31-and-a6_bld-0000000/build >cat .mozconfig ... ># here we patch the build with the tweaks we need to make it work with firefox beta >curl -L 'https://bug1040319.bugzilla.mozilla.org/attachment.cgi?id=8490842' | patch -p1 -i - You should move the patching part just after "hg up". It should be applied before you copy the used mozconfig to .mozconfig. Otherwise you'll have the same product ID.
Thanks for taking care of everything yesterday Rail! Updated version attached, in case we need it again... Thanks, Pete
Attachment #8492346 - Attachment is obsolete: true
Attachment #8494299 - Flags: review?(rail)
Flags: needinfo?(kbrosnan)
Comment on attachment 8494299 [details] bug1040319_build_script.sh Yup, exactly like that.
Attachment #8494299 - Flags: review?(rail) → review+
Pete - I think we're done here and this bug can be resolved. Can you please confirm.
Flags: needinfo?(pmoore)
Quite right! Thanks Lawrence.
Status: NEW → RESOLVED
Closed: 11 years ago
Flags: needinfo?(pmoore)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: