Hadoop编写MapReduce之统计学生平均成绩

最新推荐文章于 2024-06-12 17:06:35 发布

原创

最新推荐文章于 2024-06-12 17:06:35 发布 · 9.3k 阅读

100 ·

CC 4.0 BY-SA版权

文章标签：

#hadoop #mapreduce #eclipse #java

本文介绍如何使用Hadoop集群进行学生平均成绩的统计，包括创建和上传数据文件、编写并运行Java程序Score.java、将程序打包为jar并部署到集群中，最终展示统计结果。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

目录标题

- 统计学生的平均成绩

统计学生的平均成绩

先在集群里创建几个文件

第一个：vim score.txt

[hadoop@master mapreduce]$ cd
[hadoop@master ~]$ ls
hadoop-2.7.7.master.tar.gz hadoop-2.7.7.tar.gz

[hadoop@master ~]$ vim score.txt
linli math 95
linli chinese 90
linli english 100
liming math 78
liming chinese 86
liming english 90
me math 90
me chinese 90
me english 90

第二个：vim score1.txt

[hadoop@master ~]$ vim score1.txt
root math 67
root chinese 89
root english 78
hadoop math 90
hadoop chinese 93
hadoop english 89

文件写好就上传到分布式文件系统

[hadoop@master ~]$ hadoop fs -mkdir /score //同样我会先创建一个存放目录

[hadoop@master ~]$ hadoop fs -lsr / //由于文件有点多，我只复制了相应的文件
lsr: DEPRECATED: Please use ‘ls -R’ instead.
drwxr-xr-x - hadoop supergroup 0 2020-04-19 21:38 /data
-rw-r–r-- 3 hadoop supergroup 51 2020-04-19 21:38 /data/1.txt
-rw-r–r-- 3 hadoop supergroup 53 2020-04-19 21:38 /data/2.txt
drwxr-xr-x - hadoop supergroup 0 2020-04-19 23:45 /out-jar
-rw-r–r-- 3 hadoop supergroup 0 2020-04-19 23:45 /out-jar/_SUCCESS
-rw-r–r-- 3 hadoop supergroup 78 2020-04-19 23:45 /out-jar/part-r-00000
drwxr-xr-x - hadoop supergroup 0 2020-04-19 21:41 /out-word
-rw-r–r-- 3 hadoop supergroup 0 2020-04-19 21:41 /out-word/_SUCCESS
-rw-r–r-- 3 hadoop supergroup 78 2020-04-19 21:41 /out-word/part-r-00000
drwxr-xr-x - hadoop supergroup 0 2020-04-20 00:29 /score

[hadoop@master ~]$ hadoop fs -put score.txt score1.txt /score/
[hadoop@master ~]$ hadoop fs -lsr /
lsr: DEPRECATED: Please use ‘ls -R’ instead.
drwxr-xr-x - hadoop supergroup 0 2020-04-19 21:38 /data
-rw-r–r-- 3 hadoop supergroup 51 2020-04-19 21:38 /data/1.txt
-rw-r–r-- 3 hadoop supergroup 53 2020-04-19 21:38 /data/2.txt
drwxr-xr-x - hadoop supergroup 0 2020-04-19 23:45 /out-jar
-rw-r–r-- 3 hadoop supergroup 0 2020-04-19 23:45 /out-jar/_SUCCESS
-rw-r–r-- 3 hadoop supergroup 78 2020-04-19 23:45 /out-jar/part-r-00000
drwxr-xr-x - hadoop supergroup 0 2020-04-19 21:41 /out-word
-rw-r–r-- 3 hadoop supergroup 0 2020-04-19 21:41 /out-word/_SUCCESS
-rw-r–r-- 3 hadoop supergroup 78 2020-04-19 21:41 /out-word/part-r-00000
drwxr-xr-x - hadoop supergroup 0 2020-04-20 00:30 /score
-rw-r–r-- 3 ha