Closed Bug 1477487 Opened 6 years ago Closed 6 years ago

"./mach bootstrap" with Firefox for Android fails with java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema

Categories

(Firefox Build System :: Bootstrap Configuration, defect)

Product:
Firefox Build System
Component:
Bootstrap Configuration
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox66 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla66
Tracking Flags:
Tracking Status
firefox66 --- fixed

People

(Reporter: ato, Assigned: nalexander)

References

(Blocks 2 open bugs)

Details

Attachments

(6 files)

Bug 1477487 - Pre: Don't require outdated packages. r?agi
(deleted), text/x-phabricator-request
Details
Bug 1477487 - Pre: Bump Android SDK version.
(deleted), text/x-phabricator-request
Details
Bug 1477487 - Part 0: Make GeckoView more prominent in |mach bootstrap|. r?agi
(deleted), text/x-phabricator-request
Details
Bug 1477487 - Part 1: Look for java and pin to Java 1.8 in |mach bootstrap|; r?agi
(deleted), text/x-phabricator-request
Details
Bug 1477487 - Part 2: Make |mach bootstrap| for Android recommend toolchain clang and lld. r?agi,#firefox-build-system-reviewers
(deleted), text/x-phabricator-request
Details
Bug 1477487 - Part 3: Make |mach bootstrap| for Android install Rust toolchains. r?agi
(deleted), text/x-phabricator-request
Details
Running "./mach bootstrap" and selecting the fourth option, "Firefox for Android", fails with a java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema exception: > % ./mach bootstrap > > Note on Artifact Mode: > > Artifact builds download prebuilt C++ components rather than building > them locally. > > Artifact builds are recommended for people working on Firefox or > Firefox for Android frontends. They are unsuitable for those working > on C++ code. For more information see: > https://developer.mozilla.org/en-US/docs/Artifact_builds. > > Please choose the version of Firefox you want to build: > 1. Firefox for Desktop Artifact Mode > 2. Firefox for Desktop > 3. Firefox for Android Artifact Mode > 4. Firefox for Android > Your choice: 4 > Executing as root: sudo apt-get install autoconf2.13 build-essential nodejs python-dev python-pip python-setuptools unzip uuid zip > > Reading package lists... 100% > Reading package lists... Done > Building dependency tree... 0% > Building dependency tree... 50% > Building dependency tree > Reading state information... Done > autoconf2.13 is already the newest version (2.13-68). > build-essential is already the newest version (12.5). > uuid is already the newest version (1.6.2-1.5+b6). > python-dev is already the newest version (2.7.15-3). > python-pip is already the newest version (9.0.1-2.3). > python-setuptools is already the newest version (39.2.0-1). > unzip is already the newest version (6.0-21). > zip is already the newest version (3.0-11+b1). > nodejs is already the newest version (8.11.3-1nodesource1). > 0 upgraded, 0 newly installed, 0 to remove and 23 not upgraded. > Executing as root: sudo dpkg --add-architecture i386 > Executing as root: sudo apt-get update > > Hit:1 http://security.debian.org/debian-security buster/updates InRelease > buster/updates InRelease > 0% [1 InRelease gpgv 38.3 kB] [Connecting to ftp.de.debian.org (141.76.2.4)] > Hit:2 http://ftp.de.debian.org/debian buster InRelease > > 0% [1 InRelease gpgv 38.3 kB] > Hit:3 https://deb.nodesource.com/node_8.x buster InRelease > > 0% [1 InRelease gpgv 38.3 kB] > 0% [Working] > 0% [2 InRelease gpgv 150 kB] > 0% [Working] > 0% [3 InRelease gpgv 4,643 B] > > Reading package lists... 0% > Reading package lists... 0% > Reading package lists... 0% > Reading package lists... 37% > Reading package lists... 74% > Reading package lists... 83% > Reading package lists... 98% > Reading package lists... 98% > Reading package lists... 98% > Reading package lists... Done > Executing as root: sudo apt-get install default-jdk wget libncurses5:i386 libstdc++6:i386 > > Reading package lists... 100% > Reading package lists... Done > Building dependency tree... 0% > Building dependency tree... 50% > Building dependency tree > Reading state information... Done > default-jdk is already the newest version (2:1.10-67). > wget is already the newest version (1.19.5-1). > libstdc++6:i386 is already the newest version (8.1.0-10). > libncurses5:i386 is already the newest version (6.1+20180210-4). > 0 upgraded, 0 newly installed, 0 to remove and 23 not upgraded. > > Looks like you have the Android NDK installed at: > /home/ato/.mozbuild/android-ndk-r15c > > > Looks like you have the Android SDK installed at: > /home/ato/.mozbuild/android-sdk-linux > We will install all required Android packages. > > > We are now installing the following Android packages: > platform-tools > build-tools;26.0.2 > platforms;android-26 > extras;android;m2repository > extras;google;m2repository > emulator > > You may be prompted to agree to the Android license. You may see some of > output as packages are downloaded and installed. > > Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema > at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156) > at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75) > at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81) > at com.android.sdklib.tool.SdkManagerCli.main(SdkManagerCli.java:117) > at com.android.sdklib.tool.SdkManagerCli.main(SdkManagerCli.java:93) > Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499) > ... 5 more > Error running mach: > > ['bootstrap'] > > The error occurred in code that was called by the mach command. This is either > a bug in the called code itself or in the way that mach is calling it. > > You should consider filing a bug for this issue. > > If filing a bug, please include the full output of mach, including this error > message. > > The details of the failure are as follows: > > CalledProcessError: Command '['/home/ato/.mozbuild/android-sdk-linux/tools/bin/sdkmanager', '--package_file=/home/ato/src/gecko/python/mozboot/mozboot/android-packages.txt']' returned non-zero exit status 1 > > File "/home/ato/src/gecko/python/mozboot/mozboot/mach_commands.py", line 32, in bootstrap > bootstrapper.bootstrap() > File "/home/ato/src/gecko/python/mozboot/mozboot/bootstrap.py", line 281, in bootstrap > getattr(self.instance, 'install_%s_packages' % application)() > File "/home/ato/src/gecko/python/mozboot/mozboot/debian.py", line 127, in install_mobile_android_packages > self.ensure_mobile_android_packages() > File "/home/ato/src/gecko/python/mozboot/mozboot/debian.py", line 154, in ensure_mobile_android_packages > no_interactive=self.no_interactive) > File "/home/ato/src/gecko/python/mozboot/mozboot/android.py", line 193, in ensure_android > ensure_android_packages(sdkmanager_tool=sdkmanager_tool, no_interactive=no_interactive) > File "/home/ato/src/gecko/python/mozboot/mozboot/android.py", line 249, in ensure_android_packages > subprocess.check_call(args) > File "/usr/lib/python2.7/subprocess.py", line 190, in check_call > raise CalledProcessError(retcode, cmd)
Not sure what to check for here. Do you have any clues? I forgot to mention in the bug that the target system is Debian testing.
Flags: needinfo?(nalexander)
I'm hitting the same problem in macos now.
(In reply to Andreas Tolfsen 「:ato」 from comment #1) > Not sure what to check for here. Do you have any clues? > > I forgot to mention in the bug that the target system is Debian testing. My guess is that we're not getting the right version of the JDK, although it would be odd to get it on both Linux and macOS. ato, Pike: can you tell me what java version is being invoked? (java -version, javac -version) I'm pretty sure we require 1.8, and we might need to be more scrupulous about _checking_ the version at bootstrap time, since OS vendors seem to be moving it forward on us frequently.
Flags: needinfo?(nalexander)
Flags: needinfo?(l10n)
Flags: needinfo?(ato)
Component: Mach Core → Bootstrap Configuration
Fuchsia:src axelhecht$ which java /usr/bin/java Fuchsia:src axelhecht$ java --version java 9.0.4 Java(TM) SE Runtime Environment (build 9.0.4+11) Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode) Fuchsia:src axelhecht$ javac --version javac 9.0.4 and then there's Fuchsia:src axelhecht$ ls -l /Library/Java/ total 0 drwxr-xr-x 2 root wheel 64 Jul 15 2017 Extensions lrwxr-xr-x 1 root wheel 64 Jul 23 11:12 Home -> /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home drwxr-xr-x 5 root wheel 160 Jul 23 11:12 JavaVirtualMachines lrwxr-xr-x 1 root wheel 62 Feb 12 20:47 MacOS -> /Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/MacOS Fuchsia:src axelhecht$ /Library/Java/Home/bin/java --version Unrecognized option: --version Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. but Fuchsia:src axelhecht$ /Library/Java/Home/bin/java -version java version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
Flags: needinfo?(l10n)
> % java --version > openjdk 10.0.2 2018-07-17 > OpenJDK Runtime Environment (build 10.0.2+13-Debian-1) > OpenJDK 64-Bit Server VM (build 10.0.2+13-Debian-1, mixed mode) > % javac --version > javac 10.0.2 # update-java-alternatives -l > java-1.10.0-openjdk-amd64 1101 /usr/lib/jvm/java-1.10.0-openjdk-amd64 I thought this initially had to do with multuple Java installations on the system, but apparently not.
Flags: needinfo?(ato)
sudo update-java-alternatives --jre-headless --jre --set java-1.8.0-openjdk-amd64 helped me get past that point. We really ought to have some check for whether the correct jre is installed/active because it took me a long time to figure out what was going on here.
These are downloaded from maven.google.com by Gradle now.
Just hygiene, no reason to lag, no reason to expect changes. Depends on D16134
This is part of the larger shift toward GeckoView and away from Firefox for Android. We need GV to "show up" in these kind of onboarding tools; here's a small first step. Depends on D16135
What is happening is that distribution JRE and JDK packages roll forward, installing a different version of Java than what is expected. We don't check the version installed, so sadness ensues. Right now, we require Java 1.8 to build, but in the near future, after Android-Gradle plugin 3.2.1+, we'll be free to use later Java versions. However, Android's `sdkmanager` itself requires exactly Java 1.8. We only require `sdkmanager` to install `emulator`, really -- everything else will be fetched by Gradle -- but I don't want to unravel that right now. So let's just provide decent error messages and try to prevent the worst of the footguns. Depends on D16136
We're seeing all sorts of problems with the NDK toolchain compilers and the various versions of the NDK. In automation we build with r17b and our own toolchain clang; let's try to standardize on that. This patch is ugly because of the evolution of mozboot. Long ago, we passed arguments around and interpolated strings into function names rather than setting members with the application name and whether we're in artifact mode. The places I needed to modify didn't have the right data at the right time so I added it to the bootstrap instance. I don't have the time or energy to use the instance variables through-out: that'll have to be future follow-up. Depends on D16137

paul: on your macOS device, could you:

  • apply the patch stack from phab
  • invoke ./mach bootstrap --no-interactive --application-choice mobile_android
  • update your mozconfig as suggested, choosing a --target you care about
  • invoke ./mach build && ./mach package

and report back to me? I expect that these changes, which make using a Mozilla-produced clang that "supported configuration", will address your segfaults -- but I'd definitely like confirmation. Thank you!

Flags: needinfo?(paul)

It works! Thank you.

Flags: needinfo?(paul)

It worked for me also but ony when targeting arm-linux-androideabi

Otherwise when targeting
x86_64-linux-android I get

0:09.40 checking whether the C compiler (/Users/petru.lingurar/.mozbuild/clang/bin/clang -std=gnu99 --target=arm-linux-androideabi -fno-short-enums -fno-exceptions -L/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib -Wl,-rpath-link=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib --sysroot=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64 -Wl,--allow-shlib-undefined -gcc-toolchain /Users/petru.lingurar/.mozbuild/android-ndk-r17b/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 -fuse-ld=lld) works... no
0:09.40 configure: error: installation or configuration problem: C compiler cannot create executables.
0:09.40 DEBUG: <truncated - see config.log for full output>
0:09.40 DEBUG: configure:1360: /Users/petru.lingurar/.mozbuild/clang/bin/clang -std=gnu99 --target=arm-linux-androideabi -o conftest -fno-short-enums -fno-exceptions -isystem /Users/petru.lingurar/.mozbuild/android-ndk-r17b/sysroot/usr/include/x86_64-linux-android -isystem /Users/petru.lingurar/.mozbuild/android-ndk-r17b/sysroot/usr/include -gcc-toolchain /Users/petru.lingurar/.mozbuild/android-ndk-r17b/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 -D__ANDROID_API__=21 -L/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib -Wl,-rpath-link=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64/usr/lib --sysroot=/Users/petru.lingurar/.mozbuild/android-ndk-r17b/platforms/android-21/arch-x86_64 -Wl,--allow-shlib-undefined -gcc-toolchain /Users/petru.lingurar/.mozbuild/android-ndk-r17b/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 -fuse-ld=lld conftest.c 1>&5
0:09.40 DEBUG: configure:1357:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
0:09.40 DEBUG: main(){return(0);}
0:09.40 DEBUG: ^
0:09.40 DEBUG: 1 warning generated.
0:09.40 DEBUG: ld.lld: error: cannot open crtbegin_dynamic.o: No such file or directory
0:09.40 DEBUG: ld.lld: error: unable to find library -lgcc
0:09.40 DEBUG: ld.lld: error: unable to find library -ldl
0:09.40 DEBUG: ld.lld: error: unable to find library -lc
0:09.40 DEBUG: ld.lld: error: unable to find library -lgcc
0:09.40 DEBUG: ld.lld: error: unable to find library -ldl
0:09.40 DEBUG: ld.lld: error: cannot open crtend_android.o: No such file or directory
0:09.40 DEBUG: clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
0:09.40 DEBUG: configure: failed program was:
0:09.40 DEBUG:
0:09.40 DEBUG: #line 1355 "configure"
0:09.40 DEBUG: #include "confdefs.h"
0:09.40 DEBUG:
0:09.40 DEBUG: main(){return(0);}
0:09.41 DEBUG: configure: error: installation or configuration problem: C compiler cannot create executables.
0:09.41 ERROR: old-configure failed
0:09.44 *** Fix above errors and then restart with "./mach build"
0:09.44 make: *** [configure] Error 1

i686-linux-android or
aarch64-linux-android
gives mainly the same error but from ld.lld just

0:09.91 DEBUG: ld.lld: error: unable to find library -lgcc
0:09.91 DEBUG: ld.lld: error: unable to find library -lgcc

Also when trying a robocop test I see

▶ ./mach robocop --serve
Error running mach:

['robocop', '--serve']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ADBTimeoutError: args: /Users/petru.lingurar/.mozbuild/android-sdk-macosx/platform-tools/adb -s 2259f4e8480d7ece wait-for-device shell su 0 id; echo adb_returncode=$?, exitcode: None, stdout:

File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mochitest/mach_commands.py", line 572, in run_robocop
device_serial=device_serial)
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 237, in verify_android_device
device = _get_device(build_obj.substs, device_serial)
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozrunner/mozrunner/devices/android_device.py", line 101, in _get_device
device = ADBDevice(adb=adb_path, verbose=verbose_logging, device=device_serial)
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozdevice/mozdevice/adb.py", line 642, in init
self.shell_output("su 0 id", timeout=timeout).find(uid) != -1):
File "/Users/petru.lingurar/Projects/Clients/mozilla-unified/testing/mozbase/mozdevice/mozdevice/adb.py", line 1443, in shell_output
raise ADBTimeoutError("%s" % adb_process)

Don't really know if it's because of this patch or not. Tested on a real device. com.android.shell was granted su.
For the ADBTimeout I see there's already a ticket - https://bugzilla.mozilla.org/show_bug.cgi?id=1510059

Petru: the ADB issue is almost certainly unrelated.

The logs you post look very confused -- you say you're targeting x86_64-linux-android but the invocations look like /Users/petru.lingurar/.mozbuild/clang/bin/clang -std=gnu99 --target=arm-linux-androideabi ... (i.e., regular ARM).

Can you ensure you have exactly one --target line in your mozconfig, and then mach clobber before trying again?

Thanks!

Flags: needinfo?(petru.lingurar)
Assignee: nobody → nalexander
Status: NEW → ASSIGNED

Thanks Nick.

Indeed i686-linux-android worked now and the robocop tests also (on the emulator).
I deleted other options I had in my mozconfig, and kept only the output from bootstrap so maybe that helped.

Flags: needinfo?(petru.lingurar)
Pushed by ebalazs@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/b7b1e008be45 Pre: Don't require outdated packages. r=agi https://hg.mozilla.org/mozilla-central/rev/9c33fda84724 Pre: Bump Android SDK version. r=agi https://hg.mozilla.org/mozilla-central/rev/8b70ac3ef8d0 Part 0: Make GeckoView more prominent in |mach bootstrap|. r=agi https://hg.mozilla.org/mozilla-central/rev/b1f74fc90f37 Part 1: Look for java and pin to Java 1.8 in |mach bootstrap|; r=agi https://hg.mozilla.org/mozilla-central/rev/bbfe0cbb90b4 Part 2: Make |mach bootstrap| for Android recommend toolchain clang and lld. r=agi,froydnj,#firefox-build-system-reviewers https://hg.mozilla.org/mozilla-central/rev/dc41aecedaac Part 3: Make |mach bootstrap| for Android install Rust toolchains. r=agi
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: