Ambari自动部署Hadoop

最新推荐文章于 2025-06-18 14:36:42 发布

weixin_50801368

最新推荐文章于 2025-06-18 14:36:42 发布

阅读量1.5k

点赞数 2

CC 4.0 BY-SA版权

分类专栏： cdh

本文链接：https://blog.csdn.net/weixin_50801368/article/details/114532524

cdh 专栏收录该内容

2 篇文章

订阅专栏

本文档详细记录了如何使用Apache Ambari来搭建和管理Hadoop大数据集群。首先，由于CDH收费，作者转向Ambari，并提供了下载链接。接着，配置了环境包括hosts文件、yum源、关闭selinux和防火墙，调整文件描述符限制，实现SSH免密登录。然后，安装JDK、MariaDB并设置数据库，导入Ambari-DDL，完成Ambari Server的安装和配置。最后，启动Ambari Server并确认服务运行正常。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

前言：

年前的时候搭建了一套cdh大数据管理平台，年后发现收费了，只能重新研究一套Apache Ambari用来管理大数据了

下载地址：

https://pan.baidu.com/s/1-H_Vmf1yXDNivaO83LnVAQ ysr0

https://pan.baidu.com/s/1EaiTmCTzxerv9W3BWGFdzg kkha

https://pan.baidu.com/s/1FCQrexNC06uNZCtceCV-Og on5y

https://pan.baidu.com/s/1my_7PgZFDSUt86DZQwOieA i58z

环境配置

# vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.1   daniel.cluster.com
192.168.122.11  vm1.cluster.com
192.168.122.12  vm2.cluster.com
192.168.122.13  vm3.cluster.com
192.168.122.14  vm4.cluster.com
192.168.122.15  vm5.cluster.com

配置本地yum源

vim /etc/yum.repos.d/hadoop.repo
[hadoop]
name=hadoop
baseurl=file:////hadoop/ambari
enabled=1
gpgcheck=0
yum clean all
yum makecache
yum repolist
yum -y install httpd
ln -s /hadoop/ambari /var/www/html/
#将所有文件都拷贝到/hadoop/ambari下
systemctl start httpd
systemctl enable httpd

关闭selinux，关闭防火墙，时间同步

所有机器调大文件描述符限制

# ulimit -SHn 20480

# vim /etc/security/limits.conf
* soft nofile 20480
* hard nofile 20480

ssh免密

ssh-keygen
for i in {1..5}; do ssh-copy-id vm$i.cluster.com ; done
vim /etc/yum.repos.d/ambari.repo
[ambari]
name=ambari
baseurl=http://192.168.122.1//ambari/
enabled=1
gpgcheck=0
 for i in {1..5}; do scp /etc/yum.repos.d/ambari.repo vm$i.cluster.com:/etc/yum.repos.d/ambari.repo; done

安装jdk

 tar xf /share/hadoop/jdk-8u191-linux-x64.tar.gz -C /usr/local/
for i in {1..5}; do scp /share/hadoop/jdk-8u191-linux-x64.tar.gz vm$i.cluster.com:/root/; done
for i in {1..5}; do ssh vm$i.cluster.com tar xf /root/jdk-8u191-linux-x64.tar.gz -C /usr/local ; done
yum install mariadb mariadb-server mysql-connector-java

systemctl start mariadb
systemctl enable mariadb

mysql
create database ambari;
grant all on ambari.* to ambari@'daniel.cluster.com' identified by 'bigdata';	

flush privileges;
quit
yum install ambari-server
 mysql	
 use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
quit

ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y	输入y自定义用户
Enter user account for ambari-server daemon (root):				使用root用户
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3				选1,2需要连网下载，速度慢，直接选择3(因为前面步骤我们已经准备好了)
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/local/jdk1.8.0_191/						对应前面解压的jdk路径
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y				输入y配置数据库连接选项
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3											选择mysql/mariadb
Hostname (localhost): daniel.cluster.com					写上ambari server的主机名
Port (3306):												数据库port
Database name (ambari):										库名，和前面授权对应	
Username (ambari):											用户名,和前面授权对应	
Enter Database Password (bigdata):							密码,和前面授权对应
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql						这个警告可以无视，前面我们已经导入了数据
Proceed with configuring remote database connection properties [y/n] (y)?	y确认
Extracting system views...
ambari-admin-2.5.1.0.159.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

 ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start............................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.
lsof -i:8080
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    27096 root 1439u  IPv6 131946      0t0  TCP *:webcache (LISTEN)