org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Metaspace
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1055)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at brave.servlet.TracingFilter.doFilter(TracingFilter.java:65)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.cloud.sleuth.instrument.web.ExceptionLoggingFilter.doFilter(ExceptionLoggingFilter.java:50)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at brave.servlet.TracingFilter.doFilter(TracingFilter.java:82)
at org.springframework.cloud.sleuth.instrument.web.LazyTracingFilter.doFilter(TraceWebServletAutoConfiguration.java:138)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
jstat观测
[root@localhost order-service]# jstat -gc 25992 250 20
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
7680.0 7680.0 0.0 0.0 99328.0 79966.3 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5115 975.479 976.493
7680.0 7680.0 0.0 0.0 99328.0 79970.4 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5116 975.728 976.743
7680.0 7680.0 0.0 0.0 99328.0 79966.2 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5117 975.934 976.948
7680.0 7680.0 0.0 0.0 99328.0 79966.1 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5119 976.275 977.289
7680.0 7680.0 0.0 0.0 99328.0 79970.2 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5120 976.453 977.468
7680.0 7680.0 0.0 0.0 99328.0 79970.7 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5122 976.789 977.803
7680.0 7680.0 0.0 0.0 99328.0 79964.6 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5123 976.957 977.971
7680.0 7680.0 0.0 0.0 99328.0 79964.3 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5125 977.285 978.299
7680.0 7680.0 0.0 0.0 99328.0 79966.3 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5126 977.453 978.467
7680.0 7680.0 0.0 0.0 99328.0 79966.3 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5128 977.795 978.809
7680.0 7680.0 0.0 0.0 99328.0 79964.3 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5129 977.962 978.976
7680.0 7680.0 0.0 0.0 99328.0 79964.3 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5131 978.296 979.311
7680.0 7680.0 0.0 0.0 99328.0 79968.5 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5132 978.478 979.493
7680.0 7680.0 0.0 0.0 99328.0 79964.4 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5133 978.657 979.671
7680.0 7680.0 0.0 0.0 99328.0 84907.0 512.0 504.6 131072.0 119994.7 15104.0 13321.5 7 1.015 5134 978.857 979.872
7680.0 7680.0 0.0 0.0 99328.0 80219.2 512.0 504.6 131072.0 119996.3 15104.0 13322.2 7 1.015 5136 979.296 980.310
7680.0 7680.0 0.0 0.0 99328.0 80213.1 512.0 504.6 131072.0 119996.3 15104.0 13322.2 7 1.015 5137 979.548 980.562
7680.0 7680.0 0.0 0.0 99328.0 80709.7 512.0 504.6 131072.0 119996.3 15104.0 13322.2 7 1.015 5138 979.789 980.803
7680.0 7680.0 0.0 0.0 99328.0 80213.3 512.0 504.6 131072.0 119996.3 15104.0 13322.2 7 1.015 5139 980.027 981.041
7680.0 7680.0 0.0 0.0 99328.0 80215.4 512.0 504.6 131072.0 119996.3 15104.0 13322.2 7 1.015 5140 980.259 981.274
可能是metaspace设置的太小,启动参数改为-XX:MaxMetaspaceSize=256m,原来是128m。改完重启,再观察
[root@localhost order-service]# jstat -gc 989 250 20
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
7680.0 7680.0 0.0 0.0 99328.0 82007.7 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 82232.1 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 82325.7 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 82935.1 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 82935.1 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 82939.2 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 82939.2 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 83546.4 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 83548.4 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 83552.5 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 83552.5 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84162.5 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84162.5 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84166.5 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84166.5 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84776.0 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84776.0 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84780.1 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 84780.1 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
7680.0 7680.0 0.0 0.0 99328.0 85509.2 512.0 506.7 103552.0 98303.3 12416.0 11560.0 10 1.439 63 15.692 17.131
[root@localhost order-service]#
观察一段时间看看
2020-7-28追加:
从上图可以看到,FGC次数还是很多,调整启动参数为:
-Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m
YGC次数增多,可以接受,FGC为0