HDFS missing blocks的处理办法

原创于 2019-12-11 11:46:55 发布 · 9.9k 阅读

13 ·

CC 4.0 BY-SA版权

大数据同时被 3 个专栏收录

84 篇文章

订阅专栏

Cloudera Hadoop

30 篇文章

订阅专栏

HDP Hadoop

5 篇文章

订阅专栏

本文介绍了解决HadoopHDFS集群中遇到的missingblocks错误的步骤。通过切换到hdfs用户，使用hadoopfsck命令检查并识别损坏的块，然后通过hadoopfsck-delete命令删除这些损坏的块，最后重新执行hadoopfsck/检查集群状态，确保所有操作都已成功执行。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Hadoop集群中遇到了HDFS missing blocks的错误，处理办法如下：

切换到hdfs用户

su - hdfs

使用hadoop fsck命令检查坏的块

hadoop fsck /

输出示例如下，

...
/user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10: CORRUPT blockpool BP-175371422-172.26.130.198-1529452063491 block blk_1074702727

/user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10: MISSING 1 blocks of total size 2459 B.....................................................
....................................................................................................
....................................................................................................
....................................................................................................
....................................................................................................
....................................................................................................
.......................................................................................Status: CORRUPT
 Total size:	1314013464630 B (Total open files size: 567 B)
 Total dirs:	2381988
 Total files:	291887
 Total symlinks:		0 (Files currently being written: 6)
 Total blocks (validated):	259824 (avg. block size 5057321 B) (Total open file blocks (not validated): 5)
  ********************************
  UNDER MIN REPL'D BLOCKS:	1 (3.8487592E-4 %)
  dfs.namenode.replication.min:	1
  CORRUPT FILES:	1
  MISSING BLOCKS:	1
  MISSING SIZE:		2459 B
  CORRUPT BLOCKS: 	1
  ********************************
 Minimally replicated blocks:	259823 (99.99962 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	11432 (4.3999014 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.04398
 Corrupt blocks:		1
 Missing replicas:		68592 (7.980528 %)
 Number of data-nodes:		4
 Number of racks:		1
FSCK ended at Tue Dec 10 22:07:41 EST 2019 in 27139 milliseconds


The filesystem under path '/' is CORRUPT

使用hadoop fsck -delete删除坏的块

hadoop fsck -delete /user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10

示例输出如下，

[hdfs@datanode-1 ~]$ hadoop fsck -delete /user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

Connecting to namenode via http://namenode-1.gxgs.gov:50070/fsck?ugi=hdfs&delete=1&path=%2Fuser%2Ftrafodion%2F.hiveStats%2FHIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10
FSCK started by hdfs (auth:SIMPLE) from /172.26.130.194 for path /user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10 at Tue Dec 10 22:10:37 EST 2019
.
/user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10: CORRUPT blockpool BP-175371422-172.26.130.198-1529452063491 block blk_1074702727

/user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10: MISSING 1 blocks of total size 2459 B.Status: CORRUPT
 Total size:	2459 B
 Total dirs:	0
 Total files:	1
 Total symlinks:		0
 Total blocks (validated):	1 (avg. block size 2459 B)
  ********************************
  UNDER MIN REPL'D BLOCKS:	1 (100.0 %)
  dfs.namenode.replication.min:	1
  CORRUPT FILES:	1
  MISSING BLOCKS:	1
  MISSING SIZE:		2459 B
  CORRUPT BLOCKS: 	1
  ********************************
 Minimally replicated blocks:	0 (0.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	0 (0.0 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	0.0
 Corrupt blocks:		1
 Missing replicas:		0
 Number of data-nodes:		4
 Number of racks:		1
FSCK ended at Tue Dec 10 22:10:37 EST 2019 in 56 milliseconds


The filesystem under path '/user/trafodion/.hiveStats/HIVE.DMA.DMA_ENTTYPE_STAT_ORC-1532663517555-NOSTRIPENUMROWSESTIMATE10' is CORRUPT

重新执行hadoop fsck /检查状态

.............................................................Status: HEALTHY
 Total size:	1313724044130 B (Total open files size: 650 B)
 Total dirs:	2381918
 Total files:	291461
 Total symlinks:		0 (Files currently being written: 16)
 Total blocks (validated):	259396 (avg. block size 5064550 B) (Total open file blocks (not validated): 7)
 Minimally replicated blocks:	259396 (100.0 %)
 Over-replicated blocks:	0 (0.0 %)
 Under-replicated blocks:	11432 (4.407161 %)
 Mis-replicated blocks:		0 (0.0 %)
 Default replication factor:	3
 Average block replication:	3.0440638
 Corrupt blocks:		0
 Missing replicas:		68592 (7.9924493 %)
 Number of data-nodes:		4
 Number of racks:		1
FSCK ended at Tue Dec 10 22:11:33 EST 2019 in 25360 milliseconds


The filesystem under path '/' is HEALTHY