JavaOne 2016: Life after Modularity

An experience report
Dan Heidinga, J9 VM Interpreter Lead
Daniel_Heidinga@ca.ibm.com
@DanHeidinga
18Sept 2016
Life after modularity

Important disclaimers
 THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
 WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION
CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED.
 ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED
ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR
INFRASTRUCTURE DIFFERENCES.
 ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.
 IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT
PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.
 IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT
OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
 NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
– CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS
OR THEIR SUPPLIERS AND/OR LICENSORS
2

Who am I?
 I've been involved with virtual machine development at IBM
since 2007 and am now the J9 Virtual Machine Team Lead.
J9 is IBM's independent implementation of the JVM.
 I've represented IBM on both the JSR 292 ('invokedynamic')
and JSR 335 ('lambda') expert groups and lead J9's
implementation of both JSRs.
 Currently leading my team in the adoption and implementation
of Java 9’s headline feature: Modularity.
 I’ve also maintain the bytecode verifier and deal with various
other parts of the runtime.
3

Current JDK levels in production?
 Java 8?
 Java 7?
 Java 6?
 Java 5?
 Java 1.4?
 Have you tried Java 9 yet?
4

5
Pre-Java 9: wild west
https://www.flickr.com/photos/30456664@N03/5763126280

Why am I here?
 The purpose of this talk is to share our experience implementing Modularity.
– We track the OpenJDK builds to implement the VM-side of features in J9.
– Running 10,000s of tests, we’ve tripped many of the immediate issues
 Number one take away from this talk: Download JDK9 ea builds and test your app.
7

Migration path
 This talk is organized in code snippets.
– Small sections of code that demonstrate the problem
– Not identical to the code we hit the problem with.
– Goal is small code snippets that are easy to digest.
 Where possible, I’ve validated the issues have been experienced by others
– And have credited them when possible.
 Were possible, I’ve shown how to work around the issues
– Some don’t have workarounds
– Some the EG are still investigating design changes
8

Module system overview
9
java.base
java.something
com.user.b
com.user.a
unnamed

Internal APIs are now hidden
 There are “Java Platform SE APIs”, and then there are those useful, internal public types...
– Sometimes usage is for a defensible reason, e.g. performance, Unsafe operations,
extending the Platform
 Java 9 will not discriminate!
– New module boundaries will only export public APIs to strangers
 Your code may be clean, but how about your dependencies?
– Some types have migrated into API in later releases, e.g. Base64Encoder
 Java 8 contains a tool called jdeps that can show you the damage
$ jdeps -jdkinternals com.ibm.mq.explorer.jmsadmin_8.0.0.201502232022.jar
com.ibm.mq.explorer.jmsadmin_8.0.0.201502232022.jar -> java.naming
com.ibm.mq.explorer.jmsadmin.ui.JmsAdminCommon (com.ibm.mq.explorer.jmsadmin_8.0.0.201502232022.jar)
-> com.sun.jndi.ldap.LdapCtxFactory JDK internal API (java.naming)
com.ibm.mq.explorer.jmsadmin.ui.internal.contexts.AddJmsContextWizPage4 (com.ibm.mq.explorer.jmsadmin_8.0.0.201502232022.jar)
-> com.sun.jndi.ldap.LdapCtxFactory JDK internal API (java.naming)
Warning: JDK internal APIs are unsupported and private to JDK implementation that are
subject to be removed or changed incompatibly and could break your application.
Please modify your code to eliminate dependency on any JDK internal APIs.
For the most recent update on JDK internal API replacements, please check:
https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool

IBM XML implementation is replaced
 IBM XML implementation (XML4J, XLXPJ, XLTXE) is replaced by the OpenJDK version
 Scan your code using the WAS Migration Toolkit to find issues
– https://developer.ibm.com/wasdev/downloads/#asset/tools-
Migration_Toolkit_for_Application_Binaries
java -jar binaryAppScanner.jar <binaryInputPath> --java9xml
java -jar binaryAppScanner.jar --help --java9xml
 Scan your code using the jdeps tool included with Java 9
 A migration guide is included in the IBM Java User Guides

Just run your app…
jre/bin/java: command not found
12

The JDK directory layout has changed
 Impact: If you rely on the file layout, your application may break
 Recommendation: JDK9 early access builds are available
– Try them, feedback is critical to mitigate issues
.
├── bin
├── include
│ └── linux
├── jre
│ ├── bin
│ │ ├── classic
│ │ └── j9vm
│ ├── lib
│ │ ├── applet
│ │ ├── boot
│ │ ├── cmm
│ │ ├── deploy
│ │ ├── endorsed
│ │ ├── ext
│ │ ├── fonts
│ │ ├── management
│ │ ├── oblique-fonts
│ │ └── security
│ └── plugin
├── lib
└── properties
└── version
.
├── bin
├── conf
│ ├── management
│ └── security
├── include
│ └── linux
└── lib
├── modules
└── security
IBM Java 8 GA OpenJDK Java 9 dev

Old World (pre-JDK9) New World (JDK9)
Installs are JRE or JDK.
JDKs contain a JRE, e.g. there are two bin directories.
There is now a single Java install that may contain
development tools, or not.
Placing code in certain directories conveys rights, i.e.
endorsed, and ext dirs.
Endorsed updates become “upgradeable modules”
Extensions concept is abandoned.
Implementation is provided in various (well-known)
JAR files such as rt.jar and tools.jar
No more JARs. IDEs etc will need to learn new file
formats and locations.
Supporting paraphernalia such as timezone info and
fonts can be seen
Implementation are hidden in modules

rt.jar / tools.jar replaced by lib/modules
 In Java 8,
– Core class libraries for the platform were found in rt.jar
– Tools like javac where found in tools.jar
 In Java 9, these classes are part of the new ‘lib/modules’ jimage file
 Symptom: NoClassDefFoundError or ClassNotFoundException
 Impact: fairly low
– More likely to break build scripts than application behavior
– Hand crafted jar URLs to these files will break
– Will need to use libraries to read the jimage format instead

Goodbye ext/ & endorsed/ directories
$ java -Djava.ext.dirs=ext Main
-Djava.ext.dirs=ext is not supported. Use -classpath instead.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
$ java -Djava.endorsed.dirs=ext Main
-Djava.endorsed.dirs=ext is not supported.
Endorsed standards and standalone APIs in modular form will be
supported via the concept of upgradeable modules.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
17

Goodbye ext/ & endorsed/ directories
Options:
 Use “-cp <dirs>” instead
 Use Java 9’s new “-upgrademodulepath <dirs>” option
 Use “-Xbootclasspath/a:” to move them to the bootpath
–Only if you require elevated privileges
18

Java 8 Classloader hierarchy
19
Bootstrap Loader
Extension Loader
Application Loader
Loads classes from:
• rt.jar
• jre/lib/endorsed
• -Djava.endorsed.dirs
• -Xbootclasspath/a:
• -Xbootclasspath/p:
Loads classes from:
• jre/lib/ext
• -Djava.ext.dirs
Loads classes from:
• -cp

Java 9 Classloader hierarchy
20
Bootstrap Loader
Platform Loader
Application Loader
Loads classes from:
• lib/modules
• -Xbootclasspath/a:
Loads classes from:
• lib/modules
Loads classes from:
• -cp
• -mp
• -upgrademodulepath

Classloader invariants maintained
Bootloader classes still return null
Class.class.getClassLoader() == null
Loop to get extension / platform loader still works
ClassLoader iterator = Test.class.getClassLoader();
ClassLoader prev = null;
while (iterator != null) {
prev = iterator;
iterator = iterator.getParent();
}
prev == extension / platform loader
21

Classloader parent example
22
Confirmation: https://issues.apache.org/jira/browse/SUREFIRE-1265

Classloader parent output
 Works on Java 8, fails on Java 9
 Classloaders should delegate to the Platform (extension) loader rather than the Bootloader
 Can work around this so it runs on both 8 & 9 by using
ClassLoader.getSystemClassLoader().getParent()
or a getParent() loop to provide a different loader.
23

Classloader invariants maintained
Bootloader classes still return null
Class.class.getClassLoader() == null
Loop to get extension / platform loader still works
ClassLoader iterator = Test.class.getClassLoader();
ClassLoader prev = null;
while (iterator != null) {
prev = iterator;
iterator = iterator.getParent();
}
prev == extension / platform loader
24

Who’s affected?
25
Bootstrap Loader Platform Loader Application Loader
java.*
jdk.*
See http://openjdk.java.net/jeps/261

URLClassloader casting
 ClassCastException is the only possible outcome.
 Work around: none
– Well, nothing simple. The answer really depends on why you were casting
27

If all you wanted was URLs…
28
Confirmation: https://github.com/Mojang/LegacyLauncher/pull/11/files

getResource() APIs may return null
Java 8: jar:file:/home/vagrant/jdk8-111/jdk1.8.0_111/jre/lib/rt.jar!/java/lang/Class.class
Java 9: null

getResource doesn’t find named module resources
…. And all JDK classes are in named modules
See http://download.java.net/java/jdk9/docs/api/java/lang/ClassLoader.html

getResource hope
See http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-September/000392.html

AccessibleObject.setAccessible’s kryptonite
32
Confirmation: https://issues.apache.org/jira/browse/CAMEL-10188

InaccessibleObjectException
Exception in thread "main" java.lang.reflect.InaccessibleObjectException: Cannot make a non-public member of
class java.lang.reflect.AccessibleObject accessible
at jdk.internal.reflect.Reflection.throwInaccessibleObjectException(java.base@9-ea/Reflection.java:414)
at java.lang.reflect.AccessibleObject.checkCanSetAccessible(java.base@9-ea/AccessibleObject.java:190)
at java.lang.reflect.Method.checkCanSetAccessible(java.base@9-ea/Method.java:192)
at java.lang.reflect.Method.setAccessible(java.base@9-ea/Method.java:186)
at Test.main(Test.java:8)
33
Most likely to cause problems for frameworks / libraries

Dealing with InaccessibleObjectException in Java 8
 AccessibleObject#setAccessible’s new InaccessibleObjectException doesn’t exist in Java 8
– And can’t be caught in a try / catch block
 Work around:
– Catch RuntimeException and check if the class name is InaccessibleObjectException
 Better work around:
– Try not to use setAccessible
34

Java version: From 1.8.x to 9.x
35

Java version: From 1.8.x to 9.x (fixed)
36

Java version: From 1.8.x to 9.x (fixed)
37

Java version: From 1.8.x to 9.x (fixed 9 only)
38

Increasing readability
--add-reads <source-module>=<target-module>
39
--add-reads java.naming=jdk.naming.rmi
java.lang.IllegalAccessException: Class
com/sun/naming/internal/ResourceManager(module java.naming) can not access
class com/sun/jndi/url/rmi/rmiURLContextFactory(module jdk.naming.rmi)
because module java.naming does not read module jdk.naming.rmi
at java.lang.J9VMInternals.newInstanceImpl(java.base/Native Method)
at java.lang.Class.newInstance(java.base/Class.java:1833)
at ...

Breaking encapsulation
--add-exports <source-module>/<package>=<target-module>(,<target-module>)*
40
java.lang.IllegalAccessException: class sun.reflect.misc.Trampoline cannot access interface
com.ibm.lang.management.JvmCpuMonitorMXBean (in module java.management) because module
java.management does not export com.ibm.lang.management to unnamed module @59d80b37
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(java.management/MBeanIntrospector.java)
at com.sun.jmx.mbeanserver.PerInterface.invoke(java.management/PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(java.management/MBeanSupport.java:252)
at ….

Multiversion jar
 Support multiple java versions with a single jar using different implementations
– A single jar
– Multiple implementations
jar --create --file mr.jar -C java8_dir Test.class --release 9 -C java9_dir Test.class

Take away
Try the JDK 9 early access builds.
Run JDEPs and fix the internal usages
Report your experience on the OpenJDK mailing lists.
42

43
Legal Notice
IBM and the IBM logo are trademarks or registered trademarks of IBM Corporation, in the United States, other
countries or both.
Java and all Java-based marks, among others, are trademarks or registered trademarks of Oracle in the United
States, other countries or both.
Other company, product and service names may be trademarks or service marks of others.
THE INFORMATION DISCUSSED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND
ACCURACY OF THE INFORMATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, AND IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT
OF THE USE OF, OR OTHERWISE RELATED TO, SUCH INFORMATION. ANY INFORMATION
CONCERNING IBM'S PRODUCT PLANS OR STRATEGY IS SUBJECT TO CHANGE BY IBM WITHOUT
NOTICE.

JavaOne 2016: Life after Modularity

More Related Content

What's hot

Viewers also liked

Similar to JavaOne 2016: Life after Modularity

Recently uploaded

JavaOne 2016: Life after Modularity