Install Hortonworks Hadoop 2.6 Using Ambari
变更历史
操作 创建
责任人 日期 版本 V1.0 创建 变更内容 szdxxhb_hairy 2017/4/12 目录
1. 2.
Hadoop集群安装规划 .................................................................................................................................................................................................. 3 下载软件包 .................................................................................................................................................................................................................... 3 2.1. 下载JDK ............................................................................................................................................................................................................. 3 2.2. 下载Ambari ....................................................................................................................................................................................................... 3 2.3. 下载Hadoop ...................................................................................................................................................................................................... 3 安装操作系统 ................................................................................................................................................................................................................ 3 2.4. 最小化安装CentOS 6.6 ..................................................................................................................................................................................... 3 2.5. 配置本地YUM源 .............................................................................................................................................................................................. 4 2.6. 安装软件包 ........................................................................................................................................................................................................ 6 2.7. 修改系统配置参数 ............................................................................................................................................................................................ 7 克隆节点及设置SSH互信 ............................................................................................................................................................................................ 7 4.1. 虚机管理软件克隆 .......................................................................................................................................................... 错误！未定义书签。 4.2. 修改主机名及网卡绑定 .................................................................................................................................................. 错误！未定义书签。 4.3. 设置SSH互信.................................................................................................................................................................................................... 7 安装Ambari ................................................................................................................................................................................................................... 8 5.1. 安装和设置HTTP Server ................................................................................................................................................. 错误！未定义书签。 5.2. 准备Ambari Local Repository ........................................................................................................................................................................... 8 5.3. 修改和分发YUM Repository文件 .................................................................................................................................................................... 9 5.4. 安装Ambari Agent和Server ............................................................................................................................................................................ 9 创建和设置Hadoop集群 ........................................................................................................................................................................................... 12 6.1. 创建Hive Metastore Repository ...................................................................................................................................................................... 12 6.2. 创建Oozie Server Repository .......................................................................................................................................................................... 12 6.3. 重启Ambari Server .......................................................................................................................................................... 错误！未定义书签。 6.4. 创建集群 .......................................................................................................................................................................................................... 12 6.5. 选择安装服务 .................................................................................................................................................................................................. 17 安装问题FAQ .............................................................................................................................................................................................................. 32 7.1. 安装挂载CentOS 6.6 DVD ............................................................................................................................................................................... 32 7.2. Hive 启动失败................................................................................................................................................................. 错误！未定义书签。 7.3. Hbase启动失败 ............................................................................................................................................................... 错误！未定义书签。
3.
4.
5.
6.
7.
1. Hadoop集群安装规划
主机名 hadoop-db hadoop01 hadoop02 hadoop03 hadoop04 hadoop05 hadoop06 hadoop07 hadoop08
172.29.0.25 172.29.0.226 172.29.0.227 172.29.0.228 172.29.0.229 172.29.0.21 172.29.0.22 172.29.0.23 172.29.0.24 IP地址 DataNode DataNode DataNode DataNode DataNode DataNode DataNode DataNode 说明 Namenode/Ambari Server/Oracle 2. 下载离线软件包 2.1. 下载JDK
JDK:jdk-7u67-linux-x64.tar.gz
Oracle JDBC:ojdbc6-11.2.0.3.jar
2.2. 下载Ambari离线安装包（RedHat 6）
下载repo文件 wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3/ambari.repo -O /etc/yum.repos.d/ambari.repo 下载Ambari Server文件 http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3/ambari-2.5.0.3-centos6.tar.gz 2.3. 下载Hadoop
下载Hadoop组件 http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.0.3/HDP-2.6.0.3-centos6-rpm.tar.gz 下载HDP-UTILS组件 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6/HDP-UTILS-1.1.0.21-centos6.tar.gz 2.4. 下载CentOS6.4
CentOS-6.4-x86_64-bin-DVD1.iso CentOS-6.4-x86_64-bin-DVD2.iso 2.5. 下载依赖RPM包（依系统而定）
http://rpm.pbone.net/ 1、python-argparse-1.2.1-36.1.x86_64.rpm 3. 安装操作系统 3.1. 安装Redhat6.4
IT运维部提供机器，机器发布模板要求：
（1、关闭防火墙，且不自启动 2、selinux 为disabled）
3.2. 安装和设置HTTP Server
本文选择HADOOP-DB为Ambari Server，故需在HADOOP-DB上安装httpd [root@hadoop-db ~]# chkconfig --levels 345 httpd on [root@hadoop-db ~]# service httpd start 建立软连接 [root@hadoop-db html]# pwd /var/www/html [root@hadoop-db html]# cd /var/www/html/ [root@hadoop-db html]# ll 总用量 0 [root@hadoop-db html]# ln -s /mnt/redhat redhat [root@hadoop-db html]# ll 总用量 0 lrwxrwxrwx 1 root root 11 4月 12 14:43 redhat -> /mnt/redhat 3.3. 配置在线YUM源
加载Redhat和Centos6.4安装介质，并配置yum #mkdir –p /mnt/redhat #cd /etc/yum.repos.d #mkdir bak #mv *.repo bak #vi rhel-source.repo 建立本地私有库 [root@hadoop-db html]# yum -y install createrepo* Loaded plugins: product-id, refresh-packagekit, security, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. base | 4.0 kB 00:00 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-17.el6 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-17.el6.noarch --> Running transaction check ---> Package python-deltarpm.x86_64 0:3.5-0.5.20090913git.el6 will be installed --> Processing Dependency: deltarpm = 3.5-0.5.20090913git.el6 for package: python-deltarpm-3.5-0.5.20090913git.el6.x86_64 --> Running transaction check ---> Package deltarpm.x86_64 0:3.5-0.5.20090913git.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================================================================================================================== Package Arch Version Repository Size ==================================================================================================================================================================================================== Installing: createrepo noarch 0.9.9-17.el6 base 94 k Installing for dependencies: deltarpm x86_64 3.5-0.5.20090913git.el6 base 71 k python-deltarpm x86_64 3.5-0.5.20090913git.el6 base 27 k Transaction Summary ==================================================================================================================================================================================================== Install 3 Package(s) Total download size: 192 k Installed size: 518 k Downloading Packages: (1/3): python-deltarpm-3.5-0.5.20090913git.el6.x86_64.rpm | 27 kB 00:00 (2/3): createrepo-0.9.9-17.el6.noarch.rpm | 94 kB 00:00 (3/3): deltarpm-3.5-0.5.20090913git.el6.x86_64.rpm | 71 kB 00:00 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 7.8 MB/s | 192 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : deltarpm-3.5-0.5.20090913git.el6.x86_64 1/3 Installing : python-deltarpm-3.5-0.5.20090913git.el6.x86_64 2/3 Installing : createrepo-0.9.9-17.el6.noarch 3/3 Verifying : python-deltarpm-3.5-0.5.20090913git.el6.x86_64 1/3 Verifying : createrepo-0.9.9-17.el6.noarch 2/3 Verifying : deltarpm-3.5-0.5.20090913git.el6.x86_64 3/3 Installed: createrepo.noarch 0:0.9.9-17.el6 Dependency Installed: deltarpm.x86_64 0:3.5-0.5.20090913git.el6 python-deltarpm.x86_64 0:3.5-0.5.20090913git.el6 Complete! [root@hadoop-db html]# createrepo -v /mnt/others/ Spawning worker 0 with 1 pkgs Worker 0: reading python-argparse-1.2.1-36.1.x86_64.rpm Workers Finished Gathering worker results Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Starting other db creation: Thu Apr 13 14:29:32 2017 Ending other db creation: Thu Apr 13 14:29:32 2017 Starting filelists db creation: Thu Apr 13 14:29:32 2017 Ending filelists db creation: Thu Apr 13 14:29:32 2017 Starting primary db creation: Thu Apr 13 14:29:32 2017 Ending primary db creation: Thu Apr 13 14:29:33 2017 Sqlite DBs complete [root@hadoop-db html]# cd /mnt/others/ [root@hadoop-db others]# ll 总用量 264 -rw-r--r-- 1 root root 263604 4月 13 14:22 python-argparse-1.2.1-36.1.x86_64.rpm drwxr-xr-x 2 root root 4096 4月 13 14:29 repodata [root@hadoop-db others]# cd repodata/ 建立文件和html的目录链接 [root@hadoop-db html]# ll