CUDA Linux下查看GPU相关信息命令

最新推荐文章于 2025-05-14 08:30:31 发布

"灼灼其华"

最新推荐文章于 2025-05-14 08:30:31 发布

阅读量1k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： Deep Learning

本文链接：https://blog.csdn.net/weixin_44285715/article/details/102989705

Deep Learning 专栏收录该内容

13 篇文章

订阅专栏

本文介绍了如何使用lspci、nvidia-smi等命令监控GPU的状态，包括GPU型号、驱动版本、内存使用情况等，并提供了排查GPU占用及内存未释放等问题的方法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

命令：

lspci | grep -i nvidia

输出：

b:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)
af:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)

命令：

nvidia-smi -L

输出：

GPU 0: Tesla P100-PCIE-16GB (UUID: GPU-658852dd-63f5-582e-a34f-41b9e40d10f9)
GPU 1: Tesla P100-PCIE-16GB (UUID: GPU-8b0b68ae-d897-bf98-a8ce-6d0c020ca346)

命令：

nvidia-smi

输出：

Thu Dec  5 04:08:13 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.74       Driver Version: 418.74       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:3B:00.0 Off |                    0 |
| N/A   43C    P0    35W / 250W |   5338MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-PCIE...  Off  | 00000000:AF:00.0 Off |                  N/A |
| N/A   38C    P0    29W / 250W |     10MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

nvidia-smi命令详解：https://blog.csdn.net/u013066730/article/details/84831552

查看GPU运行状况

watch nvidia-smi

CUDA error:out of memory：

https://blog.csdn.net/keneyr/article/details/90266134

https://blog.csdn.net/eefresher/article/details/99979056

nvidia-smi 0进程，但Memory-Usage占满，GPU-Util为0%

可能的问题一：代码问题

https://blog.csdn.net/cx415462822/article/details/103564170

https://www.cnblogs.com/wowarsenal/p/5644813.html

可能的问题二：jupyter notebook 用完占着显存不释放

https://blog.csdn.net/m0_38007695/article/details/88954699

查看进程：

fuser -v /dev/nvidia*

(base) mcj@mcj:~$ sudo fuser -v /dev/nvidia*
                     用户     进程号 权限   命令
/dev/nvidia0:        root        927 F...m Xorg
                     mcj        1656 F...m gnome-shell
                     mcj       51465 F...m python3
                     mcj       51468 F...m python3
                     mcj       51475 F...m python3
                     mcj       87345 F...m jcef_helper
                     mcj       165919 F...m Web Content
                     mcj       165992 F...m Web Content
                     mcj       166118 F...m Web Content
                     mcj       166361 F...m Web Content
/dev/nvidiactl:      root        927 F...m Xorg
                     mcj        1656 F...m gnome-shell
                     mcj       51465 F...m python3
                     mcj       51468 F...m python3
                     mcj       51475 F...m python3
                     mcj       87345 F...m jcef_helper
                     mcj       165919 F...m Web Content
                     mcj       165992 F...m Web Content
                     mcj       166118 F...m Web Content
                     mcj       166361 F...m Web Content
/dev/nvidia-modeset: root        927 F.... Xorg
                     mcj        1656 F.... gnome-shell
                     mcj       87345 F.... jcef_helper
                     mcj       165919 F.... Web Content
                     mcj       165992 F.... Web Content
                     mcj       166118 F.... Web Content
                     mcj       166361 F.... Web Content
/dev/nvidia-uvm:     mcj       51465 F...m python3
                     mcj       51468 F...m python3
                     mcj       51475 F...m python3

ps -ef|grep 进程号

kill 进程号