Linux 持续监控进程资源使用情况相关命令

一、Linux 本地无界面监控：

1、jmap 监控 java 对象数量增长：

jmap -histo:live ${pid} | grep ${className}

watch -d -n 60 'jmap -histo:live ${pid} | grep ${className} | tee -a watch_jmap.log'

2、 watch 持续监控进程资源使用情况并输出到文件

每分钟记录一次进程资源使用情况，参数为 pid：

watch -d -n 60 'top -o %MEM -b -n 1 | grep 16515 | tee -a watch_mem.log'

3、 watch 持续监控进程资源使用情况，过滤内容并输出到文件

编写脚本(vim watch_top.sh)：

top -o %MEM -b -n 1 | grep "$1"| awk 'NR==1 {print " PID: "$1" \t MEM: "$6" \t %CPU: "$9"% \t %MEM: "$10"% \t TIME+: "$11" \t" strftime("%Y-%m-%d %H:%M:%S") }' | tee -a top_mem_"$1".log

每分钟记录一次进程资源使用情况，参数为 pid：

watch -d -n 60 ./watch_top.sh PID

4、查看进程启动时间以及运行时间

ps -eo pid,lstart,etime|grep 16515

二、Linux 多进程资源监控

1、Java 资源监控

#! /bin/sh
 
pids=`jps|grep -i "$1"| cut -d ' ' -f 1`
echo ----find pids: $pids
for pid in $pids
do
    echo ----PID: $pid----
    if [ $# -eq 1 ];
    then
        ps -q $pid -aux |  awk '{print $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11}' | column -t
    else
        #echo ${@:2}
        ps -q $pid ${@:2} |  awk '{print $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11}' | column -t
    fi
    echo " "
done

示例：

[root@bigdata0 scripts]# ./find_java_resources.sh "CoarseGrainedExecutorBackend"
----find pids: 29636 29637 29638 29639 29640 29641
----PID: 29636----
USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
root  29636  3.4   0.5   15242796  519360  ?    Sl    17:01  0:16  /ssdData/software/jdk1.8.0_144/bin/java
 
----PID: 29637----
USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
root  29637  3.3   0.5   15309360  536136  ?    Sl    17:01  0:16  /ssdData/software/jdk1.8.0_144/bin/java
 
----PID: 29638----
USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
root  29638  3.4   0.5   15242796  517472  ?    Sl    17:01  0:17  /ssdData/software/jdk1.8.0_144/bin/java
 
----PID: 29639----
USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
root  29639  3.2   0.5   15242796  512608  ?    Sl    17:01  0:16  /ssdData/software/jdk1.8.0_144/bin/java
 
----PID: 29640----
USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
root  29640  3.3   0.5   15242796  508664  ?    Sl    17:01  0:16  /ssdData/software/jdk1.8.0_144/bin/java
 
----PID: 29641----
USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
root  29641  3.3   0.5   15242796  513356  ?    Sl    17:01  0:16  /ssdData/software/jdk1.8.0_144/bin/java

2、watch 持续监控 Java资源

#!/bin/sh
 
# check params
if [ $# -lt 2 ]; then
  echo "Usage: $0 <process_name> <output_directory> [ps_options]"
  exit 1
fi
 
# get params
PROCESS_NAME=$1
OUTPUT_DIR=$2
shift 2
 
# create output dir
mkdir -p "$OUTPUT_DIR"
 
# find ids
pids=$(jps | grep -i "$PROCESS_NAME" | cut -d ' ' -f 1)
# echo ----find pids: $pids
 
for pid in $pids
do
  # echo ----PID: $pid----
  CURRENT_TIME=$(date +"%Y-%m-%d %H:%M:%S")
 
  if [ $# -eq 0 ]; then
    ps -q $pid -aux | awk -v current_time="$CURRENT_TIME" '{print current_time, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11}' | column -t >> "$OUTPUT_DIR/$pid"
  else
    ps -q $pid "$@" | awk -v current_time="$CURRENT_TIME" '{print current_time, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11}' | column -t >> "$OUTPUT_DIR/$pid"
  fi
 
  # echo "Output saved to $OUTPUT_DIR/$pid"
  # echo " "
done
 
free -g

示例：

[root@bigdata0 scripts]# watch  ./watch_java_resources.sh "CoarseGrainedExecutorBackend" "./logs"
[root@bigdata0 scripts]#
[root@bigdata0 scripts]#
[root@bigdata0 scripts]#
[root@bigdata0 scripts]#
[root@bigdata0 scripts]# cat logs/29636
2025-06-24  17:54:48  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:54:48  root  29636  0.6   0.5   15244844  523776  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java
2025-06-24  17:54:50  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:54:50  root  29636  0.6   0.5   15244844  523784  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java
2025-06-24  17:54:52  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:54:52  root  29636  0.6   0.5   15244844  523784  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java
2025-06-24  17:54:55  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:54:55  root  29636  0.6   0.5   15244844  523784  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java
2025-06-24  17:54:57  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:54:57  root  29636  0.6   0.5   15244844  523784  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java
2025-06-24  17:54:59  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:54:59  root  29636  0.6   0.5   15244844  523784  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java
2025-06-24  17:55:02  USER  PID    %CPU  %MEM  VSZ       RSS     TTY  STAT  START  TIME  COMMAND
2025-06-24  17:55:02  root  29636  0.6   0.5   15244844  523784  ?    Sl    17:01  0:20  /ssdData/software/jdk1.8.0_144/bin/java

3、批量查看环境变量

#! /bin/sh
 
pids=`jps -l|grep -i "$1"| cut -d ' ' -f 1`
echo ----find pids: $pids
for pid in $pids
do
    echo ----PID: $pid----
    cat /proc/$pid/environ | tr '\0' '\n'
    echo " "
done

示例：

[root@bigdata0 scripts]# ./find_java_env.sh "CoarseGrainedExecutorBackend"
----find pids: 29636 29637 29638 29639 29640 29641
----PID: 29636----
PATH=/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
HADOOP_CONF_DIR=/ssdData/hadoop-3.3.2/etc/hadoop
SPARK_PRINT_LAUNCH_COMMAND=1
SPARK_EXECUTOR_DIRS=/tmp/spark-97a7bfb7-e84f-4d6a-9e2c-9dfc99b4a0e8/executor-57c2a3fb-0a68-43bf-82f7-e88fe0e7a5ac
SPARK_ENV_LOADED=1
JRE_HOME=/ssdData/software/jdk1.8.0_144/jre

三、jvm 远程监控

1、使用 JMX 远程监控

服务端：
服务端 jvm 启动时需要添加参数：

com.sun.management.jmxremote：开启 jmx 远程监控；
com.sun.management.jmxremote.port=port：指定开启 JMX/RMI 连接的端口；除了发布用于本地访问的 RMI 连接器之外，设置这个属性，还会在私有的只读注册表发布一个额外的 RMI 连接器，这个连接器运行在 port 端口，使用jmxrmi；
com.sun.management.jmxremote.authenticate=false：关闭安全认证；
com.sun.management.jmxremote.ssl：关闭 SSL；
java.rmi.server.hostname：主机名，远程连接时，需要通过该主机名建立连接。默认为 localhost。

set JMX_OPTS=-Dcom.sun.management.jmxremote
set JMX_OPTS=%JMX_OPTS% -Dcom.sun.management.jmxremote.port=8888
set JMX_OPTS=%JMX_OPTS% -Dcom.sun.management.jmxremote.authenticate=false
set JMX_OPTS=%JMX_OPTS% -Dcom.sun.management.jmxremote.ssl=false
set JMX_OPTS=%JMX_OPTS% -Djava.rmi.server.hostname=192.168.1.2

set CLASSPATH=test.jar
java %JMX_OPTS% -classpath %CLASSPATH% org.example.StartUp

即：

java -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8888 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.1.2 -classpath test.jar  org.example.StartUp

可以通过 jps + jinfo 查看是否生效：

~$ jps
20308
22932 AgentLauncher
3428 StartUp
7780 RemoteMavenServer36
15368 test.jar
19848 Bootstrap
23548 Jps
:~$ jinfo -flags 3428
Attaching to process ID 3428, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.271-b09
Non-default VM flags: -XX:CICompilerCount=12 -XX:InitialHeapSize=536870912 -XX:+ManagementServer -XX:MaxHeapSize=8575254528 -XX:MaxNewSize=2858418176 -XX:MinHeapDeltaBytes=524288 -XX:NewSize=178782208 -XX:OldSize=358088704 -XX:+UnlockCommercialFeatures -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseFastUnorderedTimeStamps -XX:-UseLargePagesIndividualAllocation -XX:+UseParallelGC
Command line:  -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.1.2

客户端：
使用 JVisualVM 连接（也可以使用 jconsole ）。
右键 Remote 添加远程主机：
在这里插入图片描述
填入 java.rmi.server.hostname 指定的 ip：

右键添加的远程主机，添加 JMX 连接：

填写 com.sun.management.jmxremote.port 指定的端口号：

效果如下：