![ERGONOMICS ON JVM – MEMORY HEAP
• Memory > 512MB
• 25% MaxHeap
• Memory < 512MB
• Varies from 25% up to 50% MaxHeap
• jmap -histo[:live] <pid>
• Instances and bytes](https://image.slidesharecdn.com/javakubernetes2024-241011015508-0a250908/85/Considerations-when-deploying-Java-on-Kubernetes-8-320.jpg)
![ERGONOMICS ON JVM – MEMORY HEAP
• Memory > 512MB
• 25% MaxHeap
• Memory < 512MB
• Varies from 25% up to 50% MaxHeap
• jmap -histo[:live] <pid>
• Instances and bytes](https://image.slidesharecdn.com/javakubernetes2024-241011015508-0a250908/85/Considerations-when-deploying-Java-on-Kubernetes-9-320.jpg)
Considerations when deploying Java on Kubernetes
- 1. CONSIDERATIONS WHEN DEPLOYING JAVA APPLICATIONS TO KUBERNETES* * OR ANY CONTAINER BASED ENVIRONMENT @superserch
- 2. AGENDA • Kubernetes – request and limit • JVM – Threads and Memory spaces • Default Ergonomics on JVM • Conflict • Tuning recommendations • Cloud Foundry Paketo buildpacks
- 3. KUBERNETES – REQUEST AND LIMIT • Platform that provides services to deploy Pods (one or more containers) • Splits CPU and memory of Worker Nodes between all containers in that node • Uses linux cgroup v2 technology to constrain resources • CPU shared as processing power by time unit – throttling might happen • Memory involves all processes runing inside a container – it can be killed by oom • Request – what needs to be available on a node to be scheduled • Limit – Usage that Linux kernel will enforce on the container
- 4. JVM – THREADS AND MEMORY SPACES • Java Virtual Machine – Memory is managed by the JVM, not the programmer • Every Java application is multithreaded • JVM will configure itself based on the detected environment • JVM has different memory areas where it stores data • Heap memory – User data • Meta space memory – Class metadata and constant pool • Stack and DirectMemory • Not every OOME causes the JVM to crash
- 5. ERGONOMICS AND HEURISTICS ON JVM • Ergonomics: the study of working conditions, especially the design of equipment and furniture, in order to help people work more efficiently • Heuristics: a method of solving problems by finding practical ways of dealing with them, learning from past experience
- 6. ERGONOMICS ON JVM – GARBAGE COLLECTOR • Server class machine • G1GC • SerialGC • No server class machine • SerialGC
- 7. ERGONOMICS ON JVM – GARBAGE COLLECTOR • Server class machine • G1GC • SerialGC • No server class machine • SerialGC • Server class machine • Available Processors > 1 • Available Memory > 2GB (-256MB)
- 8. ERGONOMICS ON JVM – MEMORY HEAP • Memory > 512MB • 25% MaxHeap • Memory < 512MB • Varies from 25% up to 50% MaxHeap • jmap -histo[:live] <pid> • Instances and bytes
- 9. ERGONOMICS ON JVM – MEMORY HEAP • Memory > 512MB • 25% MaxHeap • Memory < 512MB • Varies from 25% up to 50% MaxHeap • jmap -histo[:live] <pid> • Instances and bytes
- 10. ERGONOMICS ON JVM – MEMORY METASPACE • This is off-heap memory • Initial reserved space: 1GB • If CompresedOpps are on MetaSpace will have a location table • Each class uses ~1K class space and ~8k non-class space per class • jcmd <pid> VM.metaspace • Can be limited with -XX:MaxMetaspaceSize and -XX:CompressedClassSpaceSize
- 11. ERGONOMICS ON JVM – MEMORY METASPACE • This is off-heap memory • Initial reserved space: 1GB • If CompresedOpps are on MetaSpace will have a location table • Each class uses ~1K class space and ~8k non-class space per class • jcmd <pid> VM.metaspace • Can be limited with -XX:MaxMetaspaceSize and -XX:CompressedClassSpaceSize
- 12. CONFLICT • Kubernetes will try to maintain the desired state • If a container goes beyond its memory limit it will be killed • If a container goes beyond its CPU Quota limit it will be throtteled • If a container does not respond to liveness readines it will be restarted • JVM will use as many threads as possible to do concurrent tasks • If too limited internal processes will be limited (GC, Default Executors) • When under load JVM might be paused by Linux kernel because CPU Quota • If memory is not set Java Heap might get funny values
- 13. TUNING RECOMMENDATIONS - CPU • Use -XX:ActiveProcessorCount to set the desired concurrency level • JVM won’t consume more CPU than its quota, but this setting will help in selecting the paralellism of internal structures • Is a good idea to know the CPU profile of your application, you could use Jconsole, jfr, or even top • Monitor kubernetes for CPU throttling of your containers
- 14. TUNING RECOMMENDATIONS – GARBAGE COLLECTORS • SerialGC • Default if less than 2 processors or less than 1792MB of Memory • No overhead because it stops the world to GC • GC Pause might be an issue if heap over 1GB or container throtteled • High tail-latency effect • -XX:+UseSerialGC
- 15. TUNING RECOMMENDATIONS – GARBAGE COLLECTORS • ParallelGC • No overhead because it stops the world to GC • GC Pause might be an issue if heap over 4GB or container throtteled • Great for batch workloads • High tail-latency effect • Configure at least 2000m cpu_limit • -XX:+UseParallelGC
- 16. TUNING RECOMMENDATIONS – GARBAGE COLLECTORS • G1GC • Default if at least 2 processors and more than 1792MB of Memory • some overhead because it marks regions that have changed • GC Pauses might be an issue if allocation rates are too high • High tail-latency effect • Configure at least 2000m cpu_limit • -XX:+UseG1GC
- 17. TUNING RECOMMENDATIONS – GARBAGE COLLECTORS • ZGC • moderated overhead because it does marking concurently with app • GC Pauses are usually under 1ms • Low tail-latency effect • JDK17+ • Since JDK23 is generational • Configure at least 2000m cpu_limit • -XX:+UseZGC
- 18. TUNING RECOMMENDATIONS – GARBAGE COLLECTORS • ShenandoahGC • moderated overhead because it does marking concurently with app • GC Pauses are usually under 10ms • Moderated tail-latency effect • JDK11+ • Configure at least 2000m cpu_limit • -XX:+UseShenandoahGC
- 19. TUNING RECOMMENDATIONS – MEMORY • It is convenient to know what your application has and does • Number of classes to be loaded • Use of ByteBuffer • Size of Live dataset • Size of work area • Number of threads
- 20. TUNING RECOMMENDATIONS – MEMORY • Set the Heap size • -Xmx • -Xmx3g • -XX:MaxRAMPercentage • -XX:MaxRamPercentage=75 • Consider than memory_limit > Heap + MetaSpace + Stack + DirectMemory + spare • Monitor Kubernetes OOM resets and adjust heap size or memory_limit
- 21. TUNING RECOMMENDATIONS – KUBERNETES • Make cpu_request = cpu_limit and memory_request = memory_limit • Kubernetes calculates scheduling of pods based on request values • Assign enough cpu so application and probes can run concurrently
- 22. TUNING RECOMMENDATIONS • What about setting -Xms = -Xmx ? • It not just reserves but commits memory for heap • Avoids heap resizing
- 23. CLOUD FOUNDRY PAKETO BUILDPACKS • Nice tool to build containers • At this moment defaults to BellSoft Liberica JRE 17 • It can pack a JRE or even a JLINK version of the selected JDK • It can configure some Application servers or Spring Boot applications • Adds a launcher layer that inspects environment and sets JVM parameters • This configuration is aimed to avoid OOM resets
- 24. CLOUD FOUNDRY PAKETO BUILDPACKS Heap Heap = Total Container Memory - Non Heap - Headroom Non Heap Non Heap = Direct Memory + Metaspace + Reserved Code cache + (Thread Stack * Thread Count ) Headroom defaults to 0
- 25. PAKETO DEFAULT VALUES • Direct Memory – 10MB • Reserved Code Cache – 240MB • Thread Stack – 1M (Assumes 250 threads running) • Metaspace – Calculated on class count (35% * 5800 + 14M)
- 26. PAKETO DEFAULT PARAMETERS • -XX:+ExitOnOutOfMemoryError • -XX:MaxDirectMemory • -Xmx • -XX:MaxMetaspaceSize • -XX:ReservedCodeCacheSize • -Xss • -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:PrintNMTStatistics
- 27. PAKETO CHANGE DEFAULTS • Calculator can be configured by passing environment variables • BPL_JVM_HEAD_ROOM (Percentage to not be used) • BPL_JVM_LOADED_CLASS_COUNT (Number of loaded classes) • BPL_JVM_THREAD_COUNT (Number of expected threads) • BPL_JVM_CLASS_ADJUSTMENT (as %, increments MaxMetaspace) • BPL_JAVA_NMT_ENABLED (true or false, avoids memory report on OOM) • JAVA_TOOL_OPTIONS (JVM launch flags, override paketo flags)
- 28. PAKETO CONCLUSION • Sets parameters that fixed size of Non heap memory areas • This can create OOMEs if not adjusted properly • Containers may fail to start if memory ~512MB or less • Once it calculates a Fixed non heap size for a container, if the container grows the only parameter that will change is -Xmx • -XX:MaxDirectMemorySize=10M might be too small, default value = Xmx, it might be changed by passing it in the JAVA_TOOL_OPTIONS