JVM hang suddenly with jdk version 1.8.0_312

[xijiu]

Please provide a brief summary of the bug

Our online Kafka Broker service suddenly hung up, and it's occurs with low probability and without any time regularity. no error and no logs

The command "jstack [pid]" no response, then I print the stack trace with command "jstack -m [pid]", and I found all threads state are BLOCKED, include VM Thread.

VM Thread stack:
ox0000ffff99f4da8c __pthread_cond_wait
ox0000ffff997c014c _ZN2os13PlatformEvent4parkEv
ox0000ffff9976f014 _ZN7Monitor5IWaitEP6Thread1
ox0000ffff9976faf0 _ZN7Monitor4waitEblb
ox0000ffff999355dc _ZN20SuspendibleThreadSet11synchronizeEv
ox0000ffff99858a68 _ZN20SafepointSynchronize5beginEv
ox0000ffff999d3124 _ZN8VMThread4loopEv
ox0000ffff999d3458 _ZN8VMThread3runEv
ox0000ffff997b8204 _ZL10java_startP6Thread
ox0000ffff99f47800 start_thread

I'm not sure whether this is a JVM bug. STS call synchronize() stuck.

Please provide steps to reproduce where possible

no crash log, the JVM stuck

Expected Results

JVM run normaly

Actual Results

JVM stuck

What Java Version are you using?

openjdk version "1.8.0_312" OpenJDK Runtime Environment (build 1.8.0_312-b07) OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)

What is your operating system and platform?

Linux on aarch64

How did you install Java?

a binary archive (tar.gz)

Did it work before?

it's occurs with low probability and without any time regularity. 
it occurs 3 times in half year (20 machines)

Did you test with the latest update version?

no

Did you test with other Java versions?

no

Relevant log output

No response

[xijiu]

BTW:

Only one thead state is IN_NATIVE, it's stack trace:

ox0000ffff99de7dd8 __GI___poll
ox0000fffed7871a3c NET_Timeout0
ox0000fffe786ec40 Java_java_net_PlainSocketImpl_socketAccept
ox0000ffff8c015628 * java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) bci:0 (Interpreted frame)
ox0000ffff8c008498 * java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) bci:7 line:409 (Interpreted frame)
ox0000ffff8c008498 * java.net.ServerSocket.implAccept(java.net.SocketImpl)
ox0000ffff8c008498 * java.net.ServerSocket.accept()
ox0000ffff8c008380 * sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept()
ox0000ffff8c008380 * sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop()

I'm not sure it is useful

thanks

[karianna]

@xijiu Please use the latest 1.8.0_382 and report back here if you sill get a stall. I assume you're on the latest patched Kafka?

[xijiu]

@xijiu Please use the latest 1.8.0_382 and report back here if you sill get a stall. I assume you're on the latest patched Kafka?

@karianna
Thanks for reply

1. Have others encountered similar problems? I've been searching on Google for a long time, but the receipt is very small. Only this one is more similar to my question: https://hotspot-dev.openjdk.java.narkive.com/03ycGzRS/jvm-hanging-when-using-g1gc-on-jdk8-b78-or-b79-linux-32-bit

2. Can you confirm that this is a bug in the 1.8.0_312 ? if so, we will focus on JDK. Because i not received any error from my application. We have a lot of JVM servers online. And change jdk version is a very difficult task, so need to be very cautious. this question occurs with low probability, the last time I encountered it was three months ago. so if verison 1.8.0_382 also exist this bug, we maybe wait a long long time.

3. Kafka version is 2.8.2, kafka is my job, So I'm quite familiar with it, It doesn't seem like a problem of Kafka itself

4. VM Thread stuck when invoke "SuspendibleThreadSet::synchronize", does the threads which can invoke "SuspendibleThreadSet::join()" are only marking threads ?

[karianna]

There are a few bug fixes between 1.8.0_131 and the latest 1.8.0_382 that pertain to hangs/crashes in the synchronization use case you're dealing with. I would recommend the upgrade in the first instance (also for other security and stability reasons).

[xijiu]

I see, I see, If there is any progress or new discovery on this issue, I will reply here immediately.

[github-actions]

We are marking this issue as stale because it has not been updated for a while. This is just a way to keep the support issues queue manageable.
It will be closed soon unless the stale label is removed by a committer, or a new comment is made.

[xijiu]

Ever since we upgraded the JDK to version 11, this issue has not occurred again. Thanks @karianna