CDH6.3.1部署

原创

已于 2022-05-23 21:04:24 修改 · 2.2k 阅读

5 ·

CC 4.0 BY-SA版权

文章标签：

#大数据

于 2022-03-18 22:26:49 首次发布

本文档详细介绍了如何在虚拟机环境下配置CDH集群，包括资源配置、免密操作、JDK和MySQL的安装、数据库创建、CM相关jar上传与安装、配置文件修改、启动服务以及解决过程中遇到的问题，如Java版本不兼容和磁盘空间不足等。最终成功启动CM服务器并完成集群部署，为后续服务安装做好准备。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

CDH部署

1.准备工作

资源有限，使用虚拟机进行如下配置

机器配置

IP	主机名	配置
192.168.130.221	CDH1	4G 2处理器 60G磁盘
192.168.130.222	CDH2	2G 1处理器 20G磁盘
192.168.130.223	CDH3	2G 1处理器 20G磁盘

建议配置

IP	主机名	配置
192.168.130.221	CDH1	8G 4处理器 80G磁盘
192.168.130.222	CDH2	4G 2处理器 50G磁盘
192.168.130.223	CDH3	4G 2处理器 50G磁盘

1.1 免密操作

cd /root/.ssh
[root@CDH1 .ssh]# ssh-keygen -t rsa
[root@CDH1 .ssh]# ssh-copy-id CDH1
[root@CDH1 .ssh]# ssh-copy-id CDH2
[root@CDH1 .ssh]# ssh-copy-id CDH3

[root@CDH2 .ssh]# ssh-keygen -t rsa
[root@CDH2 .ssh]# ssh-copy-id CDH1
[root@CDH2 .ssh]# ssh-copy-id CDH2
[root@CDH2 .ssh]# ssh-copy-id CDH3

[root@CDH3 .ssh]# ssh-keygen -t rsa
[root@CDH3 .ssh]# ssh-copy-id CDH1
[root@CDH3 .ssh]# ssh-copy-id CDH2
[root@CDH3 .ssh]# ssh-copy-id CDH3

1.2 安装JDK

使用以往安装的JDK(这样做在启动CM server的时候会遇到问题，如果想跳过问题，请看2.7目录下的重装jdk)

1.3 安装MySQL

安装后创建如下数据库

## 查看是否有MySQL服务
[root@CDH1 software]# rpm -qa|grep -i mysql
mysql-community-release-el6-5.noarch
## 如果有，先卸载
[root@CDH1 software]# rpm -e --nodeps mysql-community-release-el6-5.noarch
## 删除原有MySQL依赖
[root@CDH1 software]# yum remove mysql-libs
## 下载MySQL依赖
[root@CDH1 software]# yum install libaio
[root@CDH1 software]# yum -y install autoconf
[root@CDH1 software]# wget https://downloads.mysql.com/archives/get/p/23/file/MySQL-shared-compat-5.6.24-1.el6.x86_64.rpm
[root@CDH1 software]# wget https://downloads.mysql.com/archives/get/p/23/file/MySQL-shared-5.6.24-1.el6.x86_64.rpm
[root@CDH1 software]# rpm -ivh MySQL-shared-5.6.24-1.el6.x86_64.rpm
[root@CDH1 software]# rpm -ivh MySQL-shared-compat-5.6.24-1.el6.x86_64.rpm

安装服务端

## 安装MySQL服务端
[root@CDH1 software]# rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm
## 查看随机产生的密码
[root@CDH1 software]# cat /root/.mysql_secret
# The random password set for the root user at Sun Mar 13 16:16:04 2022 (local time): 0bVp5O9LI0UmHPo3
## 查看MySQL状态并启动
[root@CDH1 software]# service mysql status
 ERROR! MySQL is not running
[root@CDH1 software]# service mysql start
Starting MySQL.. SUCCESS!
[root@CDH1 software]#

MySQL客户端

## 安装MySQL客户端
[root@CDH1 software]# rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm
## 连接MySQL
[root@CDH1 software]# mysql -uroot -p0bVp5O9LI0UmHPo3
## 修改密码
mysql> set password=password('111213');
## 退出
mysql> exit

MySQL权限

mysql> use mysql;
mysql> select user,host from user;
mysql> update user set host='%' where host='localhost';
mysql> delete from user where host!='%';
mysql> flush privileges;
mysql> quit;

MySQL建库

1）创建各组件需要的数据库
mysql> GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm';

mysql> CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
mysql> CREATE DATABASE hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

1.4 MySQL连接jar

上传mysql-connector-java-5.1.27-bin.jar到/usr/share/java

[root@CDH1 lib]# mkdir /usr/share/java
[root@CDH1 lib]# cp mysql-connector-java.jar  /usr/share/java
[root@CDH1 lib]# cd /usr/share/java
[root@CDH1 java]# ls
mysql-connector-java.jar
[root@CDH1 java]# scp -r mysql-connector-java.jar CDH2:/usr/share/
root@cdh2's password:
mysql-connector-java.jar