Docker containers & java
What I wish I’ve been told!
Mohammed Aboullaite
@laytoun
Deputy CTO - xHub
Docker containers &
java!
Hi, I’m Fero!
Agenda
● Java in containers … a good idea ?
● Creating effective Java Docker Images: smaller, faster, secure.
● Tools and libraries: build, versioning, testing
● Going to production … what we should know !?
● Java & Docker Container features
Containers are
everywhere!
Even if you don’t run containers
… You’re in a container!
You’re always in a
container!
Docker Containers & Java
First try...
Your Java Runtime...
● Oracle JDK
● OpenJDK builds by Oracle
● AdoptOpenJDK builds
● AdoptOpenJDK OpenJ9 builds
● Linux (Red Hat, Debian, Fedora, Arch, Ubuntu) OpenJDK builds
● Azul Zulu
● Amazon Corretto
● SAP SapMachine
● Liberica from Bellsoft
https://blog.joda.org/2018/09/time-to-look-beyond-oracles-jdk.html
Image size
Java
• Java Modularity
• Code refactoring
• Dependencies
Docker
• Smaller base image
• Layers & Caching (dockerfilelint)
• Clean up right away
• Dockerignore (faster builds)
• Multistage Build
• docker-squash
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
Did someone say alpine ?!
Lightweight Linux distribution based on musl libc and busybox
Project Portola: https://openjdk.java.net/projects/portola/
Did someone say alpine ?!
Java docker image size
Distroless Container Image
“Distroless" images contain only your application and its runtime
dependencies. They do not contain package managers, shells or any
other programs you would expect to find in a standard Linux
distribution.”
● Built using Google’s bazel tool
● Provide stripped down base image
● Support for: Java, Golang, Dotnet, Node, Python, C
https://github.com/GoogleContainerTools/distroless
Why should I care ?!
● Effective tracking: Improves the signal to noise of scanners (e.g.
CVE)
● Time and cost: Faster updates, less network costs
● Security: Less components that can be exploited and smaller attack
surface
Tips and tricks from Docker captains
Fast startup
Java
• Java 11
• Class-Data Sharing: since java 5
• Ahead-Of-Time compilation: since java 9
• Native image build: Graal VM
Demo
https://github.com/aboullaite/java-docker
Containers are the
executables of the cloud!
But ...
Java
jam jam
Java
Maven
● fabric8-maven-plugin (Fabric8)
● dockerfile-maven-plugin (spotify)
● Maven exec plugin (Not elegant!)
Maven
fabric8
docker
maven
Maven
spotify docker
maven
spotify
docker
client
Maven
maven
exec
Gradle
● Docker Gradle Plugin (Benjamin Muschko)
● Docker Gradle Plugin (palantir)
● Docker Gradle Plugin (Transmode)
Gradle
palantir
gradle
docker
Gradle
Transmode
gradle docker
Docker
java
client
Gradle
Benjamin gradle
docker
Docker
java
client
Google Jib
Fast for incremental
builds
Daemonless Pure Java
Reproducible
Maven and Gradle
Support
What it does ...
FROM gcr.io/distroless/java
COPY target/dependencies /app/dependencies
COPY target/resources /app/resources
COPY target/classes /app/classes
ENTRYPOINT java -cp /app/dependencies/*:/app/resources:/app/classes my.app.Main
Image versioning
● Docker Tag command (Semver, Calver, ...)
● Append build number
○ https://www.mojohaus.org/buildnumber-maven-plugin/
● Git commit hash
○ https://github.com/git-commit-id/maven-git-commit-id-plugin
Security
● Know what in your container
● Vulnerability scanning (Docker EE, Microscanner, Clair, ...)
● Docker Content Trust
● Least privilege
○ Change USER
○ Read-only filesystems
○ Limit ressources
● Minimal container OS (Minimal attack surface)
● Monitoring & Auditing
Tips and tricks from the captains session
How about
(Integration) testing!
● Real world isolated testing
● Can be run during development
● Reproducible
● As real as possible (databases,
brokers, ...)
● Cross platform
● Spot issues before they appear in
prod
● Easy to run & maintain!!!
TestContainers
● Java library
● Supports of JVM testing framework
(JUnit, Spock, ...)
● Provides lightweight, throwaway
instances of common databases
● Port randomisation
● Containers Cleanup on JVM
shutdown
GenericContainer redis =
new GenericContainer( "redis:3.0.6")
.withExposedPorts( 6379);
Let’s go to (pre-)production!
● Petclinic: Spring App
● Modernizing legacy app with containers
● Optimizing resources
● Easy scaling
● Cloud ready
● Orchestration
Demo
https://github.com/aboullaite/java-docker
Epic fail!
A brief history of containers & ...
Container Runtime and Image Format Standards - Jeff Borek & Stephen Walli - KubeCon 2017
… & Java!
zeroturnaround - A Short History of Nearly Everything Java
https://zeroturnaround.com/rebellabs/a-short-history-of-nearly-everything-java/
JVM ergonomics
The JVM has plenty of ergonomics which are based on the underlying
system
● Memory
● # CPUs
● Exact CPU model
● Garbage collector
● JIT Optimizations
⇒ -XX:+PrintFlagsFinal -XX:+PrintGCDetails
https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/ergonomics.html
Memory
Eden Tenured
Permanent
/
Metaspace
S0 S1
Young generation Old generation Permanent
generation
Heap Off Heap
CPU
● VM internal thread pools
● fork/join pool (Streams API)
● Core.asynk
● Elasticsearch
● Tomcat
● Jetty
● Netty
● Wildfly
● ...
Runtime.getRuntime().availableProcessors()
Java 8u121 and before …
forget about it!
Fabric8 got it right
https://github.com/fabric8io-images/run-java-sh
java 9, 8u131
● -XX:ParallelGCThreads and -XX:CICompilerCount are set based on
Containers CPU limits (can be overridden)
○ calculated from --cpuset-cpus
● Memory Configuration
○ -XX:+UnlockExperimentalVMOptions
○ -XX:+UseCGroupMemoryLimitForHeap
○ set -XX:MaxRAMFraction to 2 (default is 4)
Java 10+ & 8u191 +
More container awareness
● Improve heap memory allocations [JDK-8196595]:
○ -XX:InitialRAMPercentage, -XX:MaxRAMPercentage and
-XX:MinRAMPercentage
○ -XX:InitialRAMFraction, -XX:MaxRAMFraction, and
-XX:MinRAMFraction are Deprecated
● The total number of CPUs available to the Java process is calculated from
--cpus, --cpu-shares, --cpu-quota [JDK-8146115]
○ Use -XX:-UseContainerSupport to return to the old behavior
○ # processors that the JVM will use internally -XX:ActiveProcessorCount
● Attach in linux became be relative to /proc/pid/root and namespace aware
(jcmd, jstack, etc)
Java 11
More container awareness
● Removes -XX:+UnlockExperimentalVMOptions,
-XX:+UseCGroupMemoryLimitForHeap [JDK-8194086]
● jcmd -l and jps commands do not list JVMs in Docker containers
[JDK-8193710]
● Container Metrics (-XshowSettings:system) [JDK-8204107]
● Update CPU count algorithm when both cpu shares and quotas are
used [JDK-8197867]
○ -XX:+PreferContainerQuotaForCPUCount
JVM Troubleshooting & Monitoring
● Built-in tools by JVM:
○ jstat
○ jcmd
○ jmap (Not recommended)
○ jhat …
● Expose JMX port
○ VisualVM
○ jConsole
● Micrometer
● Others: New Relic, Stackify,
AppDynamics, Dynatrace...
● Docker commands
○ stats
○ inspect
○ top
● Container aware tools
○ ctop
○ dstat
● CAdvisor
● Prometheus
● Docker EE, Datadog, Sysdig, ...
Key takeaways
● Java <3 Docker
● Start your java 11 journey (or at least 8u191+)
● Java has a rich ecosystem
● Know your tools:
○ Build Custom JRE
○ CDC
○ AOP
○ GraalVM & Substrate VM
Hallway Track
Thank you!
hallwaytrack.dockercon.com/h
allway-tracks/41300/
Wednesday May 1st
at 2:00 pm
@laytoun

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

  • 1.
    Docker containers &java What I wish I’ve been told!
  • 2.
    Mohammed Aboullaite @laytoun Deputy CTO- xHub Docker containers & java!
  • 3.
  • 4.
    Agenda ● Java incontainers … a good idea ? ● Creating effective Java Docker Images: smaller, faster, secure. ● Tools and libraries: build, versioning, testing ● Going to production … what we should know !? ● Java & Docker Container features
  • 5.
  • 6.
    Even if youdon’t run containers … You’re in a container! You’re always in a container!
  • 7.
  • 8.
  • 9.
    Your Java Runtime... ●Oracle JDK ● OpenJDK builds by Oracle ● AdoptOpenJDK builds ● AdoptOpenJDK OpenJ9 builds ● Linux (Red Hat, Debian, Fedora, Arch, Ubuntu) OpenJDK builds ● Azul Zulu ● Amazon Corretto ● SAP SapMachine ● Liberica from Bellsoft https://blog.joda.org/2018/09/time-to-look-beyond-oracles-jdk.html
  • 10.
    Image size Java • JavaModularity • Code refactoring • Dependencies Docker • Smaller base image • Layers & Caching (dockerfilelint) • Clean up right away • Dockerignore (faster builds) • Multistage Build • docker-squash https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
  • 11.
    Did someone sayalpine ?! Lightweight Linux distribution based on musl libc and busybox Project Portola: https://openjdk.java.net/projects/portola/
  • 12.
    Did someone sayalpine ?!
  • 13.
  • 14.
    Distroless Container Image “Distroless"images contain only your application and its runtime dependencies. They do not contain package managers, shells or any other programs you would expect to find in a standard Linux distribution.” ● Built using Google’s bazel tool ● Provide stripped down base image ● Support for: Java, Golang, Dotnet, Node, Python, C https://github.com/GoogleContainerTools/distroless
  • 15.
    Why should Icare ?! ● Effective tracking: Improves the signal to noise of scanners (e.g. CVE) ● Time and cost: Faster updates, less network costs ● Security: Less components that can be exploited and smaller attack surface Tips and tricks from Docker captains
  • 16.
    Fast startup Java • Java11 • Class-Data Sharing: since java 5 • Ahead-Of-Time compilation: since java 9 • Native image build: Graal VM
  • 17.
  • 18.
  • 19.
  • 20.
    Maven ● fabric8-maven-plugin (Fabric8) ●dockerfile-maven-plugin (spotify) ● Maven exec plugin (Not elegant!) Maven fabric8 docker maven Maven spotify docker maven spotify docker client Maven maven exec
  • 21.
    Gradle ● Docker GradlePlugin (Benjamin Muschko) ● Docker Gradle Plugin (palantir) ● Docker Gradle Plugin (Transmode) Gradle palantir gradle docker Gradle Transmode gradle docker Docker java client Gradle Benjamin gradle docker Docker java client
  • 22.
    Google Jib Fast forincremental builds Daemonless Pure Java Reproducible Maven and Gradle Support
  • 23.
    What it does... FROM gcr.io/distroless/java COPY target/dependencies /app/dependencies COPY target/resources /app/resources COPY target/classes /app/classes ENTRYPOINT java -cp /app/dependencies/*:/app/resources:/app/classes my.app.Main
  • 24.
    Image versioning ● DockerTag command (Semver, Calver, ...) ● Append build number ○ https://www.mojohaus.org/buildnumber-maven-plugin/ ● Git commit hash ○ https://github.com/git-commit-id/maven-git-commit-id-plugin
  • 25.
    Security ● Know whatin your container ● Vulnerability scanning (Docker EE, Microscanner, Clair, ...) ● Docker Content Trust ● Least privilege ○ Change USER ○ Read-only filesystems ○ Limit ressources ● Minimal container OS (Minimal attack surface) ● Monitoring & Auditing Tips and tricks from the captains session
  • 26.
    How about (Integration) testing! ●Real world isolated testing ● Can be run during development ● Reproducible ● As real as possible (databases, brokers, ...) ● Cross platform ● Spot issues before they appear in prod ● Easy to run & maintain!!!
  • 27.
    TestContainers ● Java library ●Supports of JVM testing framework (JUnit, Spock, ...) ● Provides lightweight, throwaway instances of common databases ● Port randomisation ● Containers Cleanup on JVM shutdown GenericContainer redis = new GenericContainer( "redis:3.0.6") .withExposedPorts( 6379);
  • 28.
    Let’s go to(pre-)production! ● Petclinic: Spring App ● Modernizing legacy app with containers ● Optimizing resources ● Easy scaling ● Cloud ready ● Orchestration
  • 29.
  • 30.
  • 31.
    A brief historyof containers & ... Container Runtime and Image Format Standards - Jeff Borek & Stephen Walli - KubeCon 2017
  • 32.
    … & Java! zeroturnaround- A Short History of Nearly Everything Java https://zeroturnaround.com/rebellabs/a-short-history-of-nearly-everything-java/
  • 33.
    JVM ergonomics The JVMhas plenty of ergonomics which are based on the underlying system ● Memory ● # CPUs ● Exact CPU model ● Garbage collector ● JIT Optimizations ⇒ -XX:+PrintFlagsFinal -XX:+PrintGCDetails https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/ergonomics.html
  • 34.
    Memory Eden Tenured Permanent / Metaspace S0 S1 Younggeneration Old generation Permanent generation Heap Off Heap
  • 35.
    CPU ● VM internalthread pools ● fork/join pool (Streams API) ● Core.asynk ● Elasticsearch ● Tomcat ● Jetty ● Netty ● Wildfly ● ... Runtime.getRuntime().availableProcessors()
  • 36.
    Java 8u121 andbefore … forget about it!
  • 37.
    Fabric8 got itright https://github.com/fabric8io-images/run-java-sh
  • 38.
    java 9, 8u131 ●-XX:ParallelGCThreads and -XX:CICompilerCount are set based on Containers CPU limits (can be overridden) ○ calculated from --cpuset-cpus ● Memory Configuration ○ -XX:+UnlockExperimentalVMOptions ○ -XX:+UseCGroupMemoryLimitForHeap ○ set -XX:MaxRAMFraction to 2 (default is 4)
  • 39.
    Java 10+ &8u191 + More container awareness ● Improve heap memory allocations [JDK-8196595]: ○ -XX:InitialRAMPercentage, -XX:MaxRAMPercentage and -XX:MinRAMPercentage ○ -XX:InitialRAMFraction, -XX:MaxRAMFraction, and -XX:MinRAMFraction are Deprecated ● The total number of CPUs available to the Java process is calculated from --cpus, --cpu-shares, --cpu-quota [JDK-8146115] ○ Use -XX:-UseContainerSupport to return to the old behavior ○ # processors that the JVM will use internally -XX:ActiveProcessorCount ● Attach in linux became be relative to /proc/pid/root and namespace aware (jcmd, jstack, etc)
  • 40.
    Java 11 More containerawareness ● Removes -XX:+UnlockExperimentalVMOptions, -XX:+UseCGroupMemoryLimitForHeap [JDK-8194086] ● jcmd -l and jps commands do not list JVMs in Docker containers [JDK-8193710] ● Container Metrics (-XshowSettings:system) [JDK-8204107] ● Update CPU count algorithm when both cpu shares and quotas are used [JDK-8197867] ○ -XX:+PreferContainerQuotaForCPUCount
  • 41.
    JVM Troubleshooting &Monitoring ● Built-in tools by JVM: ○ jstat ○ jcmd ○ jmap (Not recommended) ○ jhat … ● Expose JMX port ○ VisualVM ○ jConsole ● Micrometer ● Others: New Relic, Stackify, AppDynamics, Dynatrace... ● Docker commands ○ stats ○ inspect ○ top ● Container aware tools ○ ctop ○ dstat ● CAdvisor ● Prometheus ● Docker EE, Datadog, Sysdig, ...
  • 42.
    Key takeaways ● Java<3 Docker ● Start your java 11 journey (or at least 8u191+) ● Java has a rich ecosystem ● Know your tools: ○ Build Custom JRE ○ CDC ○ AOP ○ GraalVM & Substrate VM
  • 43.