HadoopHA 搭建

1.准备三台机器 centos7

安装步骤 略（ 可以看我的 虚拟机安装 centos7 文章）

2.配置免秘钥访问

略（可以看我的免秘钥配置文章）

3.JKD 安装1.8

略

4.Hadoop官网下载3.1.2

网址 http://hadoop.apache.org/
https://archive.apache.org/dist/hadoop/common/hadoop-3.1.2/

5.解压

tar -zxvf hadoop-3.1.2.tar.gz
 mv hadoop-3.1.2 /opt/test/
 cd /opt/test/hadoop-3.1.2/etc/hadoop/

6.修改环境变量

• vi hadoop-env.sh

exportJAVA_HOME=/usr/lib/jvm/jdk1.8.0_111
exportHDFS_NAMENODE_USER=root
exportHDFS_DATANODE_USER=root
exportHDFS_ZKFC_USER=root
exportHDFS_JOURNALNODE_USER=root
exportYARN_RESOURCEMANAGER_USER=root
exportYARN_NODEMANAGER_USER=root

7.修改配置文件

• vi core-site.xml

<property><name>fs.defaultFS</name><value>hdfs://hdfs-test</value></property><property><name>hadoop.tmp.dir</name><value>/var/test/hadoop/ha</value></property><property><name>hadoop.http.staticuser.user</name><value>root</value></property><property><name>ha.zookeeper.quorum</name><value>10.1.1.201:2181,10.1.1.158:2181,10.1.1.185:2181</value></property>

• vi hdfs-site.xml

<property><name>dfs.nameservices</name><value>hdfs-test</value></property><property><name>dfs.ha.namenodes.hdfs-test</name><value>nn1,nn2</value></property><property><name>dfs.namenode.rpc-address.hdfs-test.nn1</name><value>10.1.1.58:8020</value></property><property><name>dfs.namenode.rpc-address.hdfs-test.nn2</name><value>10.1.1.152:8020</value></property><property><name>dfs.namenode.http-address.hdfs-test.nn1</name><value>10.1.1.58:9870</value></property><property><name>dfs.namenode.http-address.hdfs-test.nn2</name><value>10.1.1.152:9870</value></property><property><name>dfs.namenode.shared.edits.dir</name><value>qjournal://10.1.1.58:8485;10.1.1.152:8485;10.1.1.195:8485/hdfs-test</value></property><property><name>dfs.journalnode.edits.dir</name><value>/var/test/hadoop/ha/qjm</value></property><property><name>dfs.client.failover.proxy.provider.hdfs-test</name><value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxy
Provider</value></property><property><name>dfs.ha.fencing.methods</name><value>sshfence</value><value>shell(true)</value></property><property><name>dfs.ha.fencing.ssh.private-key-files</name><value>/root/.ssh/id_rsa</value></property><property><name>dfs.ha.automatic-failover.enabled</name><value>true</value></property><property><name>dfs.replication</name><value>2</value></property>

• vi workers
  

10.1.1.58
10.1.1.152
10.1.1.195

8.分发

scp -r hadoop-3.1.2 [email protected]:/opt/test
scp -r hadoop-3.1.2 [email protected]:/opt/test

9.修改环境变量

• vi /etc/profile

exportHADOOP_HOME=/opt/test/hadoop-3.1.2
exportPATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

分发

• scp /etc/profile root@10.1.1.152:/etc/profile
• scp /etc/profile root@10.1.1.195:/etc/profile
• source /etc/profile

10.首先启动Zookeeper

三节点都启动

[root@node01 ]#zkServer.sh start[root@node02 ]#zkServer.sh start[root@node03 ]#zkServer.sh start
zkServer.sh status

11.启动JournalNode

三节点都启动

[root@node01 ]# hdfs --daemon start journalnode[root@node02 ]# hdfs --daemon start journalnode[root@node03 ]# hdfs --daemon start journalnode

12.格式化NameNode

[root@node01 ]# hdfs namenode -format[root@node01 ]# hdfs --daemon start namenode[root@node02 ]# hdfs namenode -bootstrapStandby[root@node01 ]# hdfs zkfc -formatZK[root@node01 ]# start-dfs.sh

13.测试

http://10.1.1.58:9870
http://10.1.1.152:9870