VMware 搭建 hadoop 完全分布式集群

1. 虚拟机安装、克隆

使用VMware安装CentOS6.8详细教程

2. 网络配置

# 修改 hostname
hostname hadoop1

hostnamectl set-hostname hadoop1

# 最靠谱方式
Vim /etc/sysconfig/network

打开网卡：

centos

安装后默认是关闭网卡的，要进

/etc/sysconfig/network-scripts/

下修改

ifcfg-ens33

，将

ONBOOT

的值改成

yes

。ONBOOT是指明在系统启动时是否激活网卡，只有在激活状态的网卡才能连接网络，进行网络通信

修改完毕后，

service network restart

重启网络，检查

ping baidu.com

是否能够通

克隆虚拟机修改 onboot 后重启网络失败

错误类型：Bringing up interface eth0:Device eth0 does not seem to be present,delaying
解决方法：删除 /etc/udev/rules.d/70-persistent-net.rules，然后将网卡配置文件 /etc/sysconfig/network-scripts/ifcfg-eth0 的 uuid 和 hwaddr 这两行删除，重启系统即可！
参考：https://jingyan.baidu.com/article/e75aca85006645142edac6df.html

注意：有些没有
ifcfg-ens33
，只有
ifcfg-eth0
，修改
ifcfg-eth0
即可！

2.1 设置静态 IP、主机映射

设置静态 IP

以下操作每台服务器

hadoop1/2/3

都需要设置：

1、

Vim /etc/sysconfig/network-scripts/ifcfg-ens33

，有些可能会是

ifcfg-eth0

：

TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static       # 设置为 静态
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=1c71c1dc-e4f8-4594-b77e-5e04f6906a31
DEVICE=ens33
ONBOOT=yes            # 必须为 yes
IPADDR=192.168.131.137# 静态 IP    
GATEWAY=192.168.2.2# 网关
NETMAK=255.255.255.0# 子网掩码
DNS1=8.8.8.8
DNS2=114.114.114.114

2、每一台虚拟机配置

ip

和域名映射：

# vim /etc/hosts192.168.131.137 hadoop1
192.168.131.138 hadoop2
192.168.131.139 hadoop3

3、关闭防火墙：

# 关闭防火墙
Service iptables stop 

# 禁止防火墙开机自启
Chkconfig iptables off

# 关闭selinux，设置 SELINUX 参数为 disabled
Vim /etc/selinux/config

4、重启网络、检查：

# 重启网络
service network restart

# hadoop1/2/3 三台服务器之间相互 ping 看是否能够通
ping hadoop1    

# 重启
reboot

2.2 添加hadoop账户并分配sudo权限

useradd -m hadoop
passwd hadoop

# 分配 sudo 权限，在该行root ALL=(ALL) ALL下添加hadoop ALL=(ALL) ALL保存后退出，并切换回hadoop用户
visudo

# 如下配置## The COMMANDS section may have other options added to it.#### Allow root to run any commands anywhere
root    ALL=(ALL)       ALL
hadoop  ALL=(ALL)       ALL

# 切换用户
su hadoop

三台服务器IP、用户名、密码对应

# 分别有两个用户：root（110139）、hadoop（hadoop）192.168.131.137 hadoop1
192.168.131.138 hadoop2
192.168.131.139 hadoop3

2.3 实现三台服务器之间免密登录

1、安装

ssh

服务：

yum install -y openssl openssh-server
yum -y install openssh-clients            # 一定要安装客户端，不然使用不了 ssh# 用vim打开配置文件/etc/ssh/sshd_config# 将 PermitRootLogin，RSAAuthentication，PubkeyAuthentication 的注释设置打开# 常用命令
service sshd restart # 重启SSH服务
service sshd start # 启动服务
service sshd stop  # 停止服务
netstat -antp | grep sshd     # 查看是否启动22端口

chkconfig sshd on        # 设置开机启动
chkconfig sshd off    # 设置禁止开机启动

yum install

时出现：

[root@hadoop3 .ssh]# yum install -y openssl openssh-server
Loaded plugins: fastestmirror
Setting up Install Process
Loading mirror speeds from cached hostfile
YumRepo Error: All mirror URLs are not using ftp, http[s]orfile.
 Eg. Invalid release/repo/arch combination/
removing mirrorlist with no valid mirrors:/var/cache/yum/x86_64/6/base/mirrorlist.txt
Error: Cannot find a valid baseurl for repo: base

解决方法：

sed -i "s|enabled=1|enabled=0|g"/etc/yum/pluginconf.d/fastestmirror.conf
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
curl -o /etc/yum.repos.d/CentOS-Base.repo https://www.xmpan.com/Centos-6-Vault-Aliyun.repo 
    
yum clean all
yum makecache
yum repolist        # 查看当前源

参考：YumRepo Error: All mirror URLs are not using ftp, http[s] or file.centos6 yum失败！

2、生成公钥私钥（一定要在 hadoop 用户下执行！！！，否则后续 start-dfs.sh 没有权限）

# 一路回车即可 路径：/home/hadoop/.ssh/id_rsa、id_rsa.pub
ssh-keygen -t rsa

# 在 hadoop1  服务器
cd /home/hadoop/.ssh/
cat id_rsa.pub > authorized_keys

# 再将 Hadoop2、Hadoop3 的 id_rsa.pub 的内容拷贝到 Hadoop1 的 authorized_keys，最后将 authorized_keys 分别传到 Hadoop2、Hadoop3 的 /home/hadoop/.ssh/
scp authorized_keys root@hadoop2:/home/hadoop/.ssh/
scp authorized_keys root@hadoop3:/home/hadoop/.ssh/# 分别给三个节点 修改authorized_keys权限
chmod 600 authorized_keys
chmod 700.ssh

# 测试，ssh + 主机名
ssh hadoop2

# hbase 集群
https://www.cnblogs.com/qingyunzong/p/8668880.html

# hadoop 集群
https://www.cnblogs.com/qingyunzong/p/8496127.html

3. jdk 安装

1、如果你使用root用户进行安装。

vi /etc/profile

即可系统变量

2、如果你使用普通用户进行安装。

vi ~/.bashrc

用户变量

# .bashrc# Source global definitionsif[-f /etc/bashrc ]; then
    ./etc/bashrc
fi

# User specific aliases and functions
export JAVA_HOME=/home/hadoop/apps/jdk1.8.0_261
export PATH=$JAVA_HOME/bin:$PATH

检查：

[hadoop@hadoop3 apps]$ source ~/.bashrc
[hadoop@hadoop3 apps]$ echo $JAVA_HOME
/home/hadoop/apps/jdk1.8.0_261[hadoop@hadoop3 apps]$ java -version
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM)64-Bit Server VM (build 25.261-b12, mixed mode)

4. hadoop 安装

集群规划：
服务/主机名IP用户HDFSYARNhadoop1192.168.131.137hadoopNameNode,DataNodeNodeManager,ResourceManagerhadoop2192.168.131.138hadoopDataNodeNodeManagerhadoop3192.168.131.139hadoopDataNodeNodeManager服务主节点从节点HDFSNameNodeDataNodeYARNResourceManagerNodeManager

注意：hadoop 用户最好给 apps、data、hadoop-2.7.5/ 等都添加权限 sudo chown hadoop:hadoop apps/

1、解压：

[hadoop@hadoop1 apps]$ tar -zxvf hadoop-2.7.5.tar.gz

2、

hadoop-env.sh

：

配置文件目录：

/home/hadoop/apps/hadoop-2.7.5/etc/hadoop

cd /home/hadoop/apps/hadoop-2.7.5/etc/hadoop
vim hadoop-env.sh

# 只需修改 jdk 路径
export JAVA_HOME=/usr/local/jdk1.8.0_73

3、

core-site.xml

：

<configuration><property><name>fs.defaultFS</name><value>hdfs://hadoop1:9000</value></property><property><name>hadoop.tmp.dir</name><value>/home/hadoop/data/hadoopdata</value></property></configuration>

4、

hdfs-site.xml

：

<configuration><property><name>dfs.namenode.name.dir</name><value>/home/hadoop/data/hadoopdata/name</value><description>为了保证元数据的安全一般配置多个不同目录</description></property><property><name>dfs.datanode.data.dir</name><value>/home/hadoop/data/hadoopdata/data</value><description>datanode 的数据存储目录</description></property><property><name>dfs.replication</name><value>2</value><description>HDFS 的数据块的副本存储个数, 默认是3</description></property><property><name>dfs.secondary.http.address</name><value>hadoop3:50090</value><description>secondarynamenode 运行节点的信息，和 namenode 不同节点</description></property></configuration>

5、

mapred-site.xml

：

<configuration><property><name>mapreduce.framework.name</name><value>yarn</value></property></configuration>

6、

yarn-site.xml

：

<configuration><!-- Site specific YARN configuration properties --><property><name>yarn.resourcemanager.hostname</name><value>hadoop1</value>// 这个必须与 resourcemanager 服务器一致，我这里是 hadoop1
        </property><property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value><description>YARN 集群为 MapReduce 程序提供的 shuffle 服务</description></property></configuration>

7、

slaves

：

vim slaves 

hadoop1
hadoop2
hadoop3

8、将

hadoop-2.7.5/

分发到其他集群：

scp -r hadoop-2.7.5/ hadoop2:/home/hadoop/apps
scp -r hadoop-2.7.5/ hadoop3:/home/hadoop/apps

9、添加环境变量（每台服务器都需要添加）：

[hadoop@hadoop1 apps]$ vim ~/.bashrc 
export HADOOP_HOME=/home/hadoop/apps/hadoop-2.7.5
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:[hadoop@hadoop1 apps]$ source ~/.bashrc 
[hadoop@hadoop1 apps]$ hadoop version
Hadoop 2.7.5
Subversion https://[email protected]/repos/asf/hadoop.git -r 18065c2b6806ed4aa6a3187d77cbe21bb3dba075
Compiled by kshvachk on 2017-12-16T01:06Z
Compiled with protoc 2.5.0
From source with checksum 9f118f95f47043332d51891e37f736e9
This command was run using /home/hadoop/apps/hadoop-2.7.5/share/hadoop/common/hadoop-common-2.7.5.jar

10、格式化：

[hadoop@hadoop1 bin]$ hadoop namenode -format21/06/1314:34:26 INFO util.GSet: Computing capacity formap NameNodeRetryCache
21/06/1314:34:26 INFO util.GSet: VM type=64-bit
21/06/1314:34:26 INFO util.GSet:0.029999999329447746%max memory 966.7 MB =297.0 KB
21/06/1314:34:26 INFO util.GSet: capacity      =2^15=32768 entries
21/06/1314:34:27 INFO namenode.FSImage: Allocated new BlockPoolId: BP-57205935-192.168.131.137-162356606709421/06/1314:34:27 INFO common.Storage: Storage directory /home/hadoop/data/hadoopdata/name has been successfully formatted.21/06/1314:34:27 INFO namenode.FSImageFormatProtobuf: Saving image file/home/hadoop/data/hadoopdata/name/current/fsimage.ckpt_0000000000000000000 using no compression
21/06/1314:34:27 INFO namenode.FSImageFormatProtobuf: Image file/home/hadoop/data/hadoopdata/name/current/fsimage.ckpt_0000000000000000000 of size 322bytes saved in0 seconds.21/06/1314:34:28 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >=021/06/1314:34:28 INFO util.ExitUtil: Exiting with status 021/06/1314:34:28 INFO namenode.NameNode: SHUTDOWN_MSG:/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hadoop1/192.168.131.137************************************************************/

格式化 Namenode 时不能创建目录

java.io.IOException: Cannot create directory /home/hadoop/data/hadoopdata/na
    
# 解决方法
https://blog.csdn.net/qq_40414738/article/details/99544777

start-dfs.sh 时没有权限

[hadoop@hadoop1 sbin]$ sudo start-dfs.sh
[sudo] password for hadoop: 
sudo: start-dfs.sh: command not found
[hadoop@hadoop1 sbin]$ start-dfs.sh
Starting namenodes on [hadoop1]
hadoop@hadoop1's password: 
hadoop1: namenode running as process 4090. Stop it first.
hadoop@hadoop2's password: hadoop@hadoop1's password: hadoop@hadoop3's password: 
hadoop2: mkdir: cannot create directory `/home/hadoop/apps/hadoop-2.7.5/logs': Permission denied
hadoop2: chown: cannot access `/home/hadoop/apps/hadoop-2.7.5/logs': No such fileor directory
hadoop2: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out
hadoop2:/home/hadoop/apps/hadoop-2.7.5/sbin/hadoop-daemon.sh: line 159:/home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out: No such fileor directory
hadoop2: head: cannot open `/home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out' for reading: No such fileor directory
hadoop2:/home/hadoop/apps/hadoop-2.7.5/sbin/hadoop-daemon.sh: line 177:/home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out: No such fileor directory
hadoop2:/home/hadoop/apps/hadoop-2.7.5/sbin/hadoop-daemon.sh: line 178:/home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out: No such fileor directory

解决办法：https://www.cnblogs.com/zknublx/p/8066693.html

开启 hdfs

[hadoop@hadoop1 sbin]$ start-dfs.sh 
Starting namenodes on [hadoop1]
hadoop1: starting namenode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-namenode-hadoop1.out
hadoop3: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop3.out
hadoop2: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out
hadoop1: datanode running as process 4822. Stop it first.
Starting secondary namenodes [hadoop3]
hadoop3: starting secondarynamenode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-secondarynamenode-hadoop3.out
[hadoop@hadoop1 sbin]$ jps
4822 DataNode
5465 NameNode
5724 Jps

开启 yarn

[hadoop@hadoop1 sbin]$ start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /home/hadoop/apps/hadoop-2.7.5/logs/yarn-hadoop-resourcemanager-hadoop1.out
hadoop3: starting nodemanager, logging to /home/hadoop/apps/hadoop-2.7.5/logs/yarn-hadoop-nodemanager-hadoop3.out
hadoop2: starting nodemanager, logging to /home/hadoop/apps/hadoop-2.7.5/logs/yarn-hadoop-nodemanager-hadoop2.out
hadoop1: starting nodemanager, logging to /home/hadoop/apps/hadoop-2.7.5/logs/yarn-hadoop-nodemanager-hadoop1.out

查看集群 jps 进程

[hadoop@hadoop1 sbin]$ jps
5891 Jps
4822 DataNode
5465 NameNode
5770 ResourceManager
5870 NodeManager

[hadoop@hadoop2 ~]$ jps
2643 NodeManager
2532 DataNode
2717 Jps

[hadoop@hadoop3 ~]$ jps
2816 DataNode
2994 NodeManager
3091 Jps

web 管理界面

HDFS : http://192.168.131.137:50070
YARN ： http://192.168.131.137:8088/cluster

https://www.cnblogs.com/qingyunzong/p/8496127.html
https://www.cnblogs.com/qingyunzong/p/8668880.html
https://blog.csdn.net/Beans___Lee/article/details/105265244
https://blog.csdn.net/qq_43106863/article/details/101530233
https://blog.csdn.net/ASN_forever/article/details/80701003
https://blog.csdn.net/weixin_41552767/article/details/107221454
https://segmentfault.com/a/1190000038229319
VMware 搭建 hadoop 完全分布式集群

5. zookeeper 集群安装

这里规划三台服务器：

hadoop1,hadoop2,hadoop3

下载地址：http://mirrors.hust.edu.cn/apache/ZooKeeper/

此处使用的是3.4.10版本

1、解压：

[hadoop@hadoop1 ~]$ cd /home/hadoop/apps
[hadoop@hadoop1 ~]$ tar -zxvf zookeeper-3.4.10.tar.gz -C apps/

2、修改配置文件：

[hadoop@hadoop1 zookeeper-3.4.10]$ cd conf/[hadoop@hadoop1 conf]$ mv zoo_sample.cfg zoo.cfg
[hadoop@hadoop1 conf]$ vim zoo.cfg 

# dataDir  内存数据库快照存放地址
dataDir=/home/hadoop/data/zkdata/# 末尾添加，配置ZK监听客户端连接的端口
dataLogDir=/home/hadoop/log/zklog        # 事务日志

server.1=hadoop1:2888:3888
server.2=hadoop2:2888:3888
server.3=hadoop3:2888:3888# server.serverid=host:tickpot:electionport   # server：固定写法# serverid：每个服务器的指定ID（必须处于1-255之间，必须每一台机器不能重复）# host：主机名# tickpot：心跳通信端口# electionport：选举端口

3、将配置文件分发到集群其他机器中：

[hadoop@hadoop1 bin]$ scp -r zookeeper-3.4.10/ hadoop2:/home/hadoop/apps
[hadoop@hadoop1 bin]$ scp -r zookeeper-3.4.10/ hadoop3:/home/hadoop/apps

注意：以下操作 4、5、6 每台服务器都需要这样操作！

4、创建日志目录、配置服务器 ID：

[hadoop@hadoop1 bin]$ mkdir -p /home/hadoop/log
[hadoop@hadoop1 bin]$ sudo chown hadoop:hadoop log/# 给 hadoop 用户添加权限[hadoop@hadoop1 bin]$ mkdir -p /home/hadoop/data/zkdata        
[hadoop@hadoop1 ~]$ cd /home/hadoop/data/zkdata
[hadoop@hadoop1 ~]$ echo 1> myid        # myid,里面存放的内容就是服务器的 id,就是 server.1=hadoop01:2888:3888 当中的 id, 就是 1

5、配置环境变量：

vim ~/.bashrc

export ZOOKEEPER_HOME=/home/hadoop/apps/zookeeper-3.4.10
export PATH=$PATH:$ZOOKEEPER_HOME/bin

source ~/.bashrc

6、启动

zkserver

服务，检查：

启动：zkServer.sh start
停止：zkServer.sh stop
查看状态：zkServer.sh status

ps -aux | grep 'zookeeper'

[hadoop@hadoop1 bin]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config:/home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop1 bin]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config:/home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.[hadoop@hadoop1 bin]$ jps
2769 Jps
2723 QuorumPeerMain

[hadoop@hadoop2 bin]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config:/home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop2 bin]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config:/home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader
[hadoop@hadoop2 bin]$ jps

[hadoop@hadoop3 bin]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config:/home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop3 bin]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config:/home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower
[hadoop@hadoop3 bin]$ jps
1898 QuorumPeerMain
1951 Jps

参考：https://www.cnblogs.com/qingyunzong/p/8619184.html

6. HBase 集群搭建

[hadoop@hadoop1 apps]$ tar -zxvf hbase-1.2.6-bin.tar.gz
[hadoop@hadoop1 apps]$ cd hbase-1.2.6/conf/# 配置文件目录[hadoop@hadoop1 conf]$ vim hbase-env.sh 

# 修改 jdk 路径 和 zookeeper 
export JAVA_HOME=/home/hadoop/apps/jdk1.8.0_261# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=false

1、修改

hbase-site.xml

：

<configuration><property><!-- 指定 hbase 在 HDFS 上存储的路径 --><name>hbase.rootdir</name><value>hdfs://hadoop1:9000/hbase</value></property><property><!-- 指定 hbase 是分布式的 --><name>hbase.cluster.distributed</name><value>true</value></property><property><!-- 指定 zk 的地址，多个用“,”分割 --><name>hbase.zookeeper.quorum</name><value>hadoop1:2181,hadoop2:2181,hadoop3:2181</value></property><property><name>hbase.master.maxclockskew</name><value>180000</value><description>Time difference of regionserver from master</description></property></configuration>

2、修改

regionservers

：

vim regionservers

hadoop1
hadoop2
hadoop3

3、修改

backup-masters

（该文件是不存在的，先自行创建）：

# 备用 master
vim backup-masters

hadoop3

4、要把 hadoop 的 hdfs-site.xml 和 core-site.xml 放到 hbase-1.2.6/conf 下

5、删除

docs

（删除

HBase

目录下的

docs

文件夹）：

rm -rf docs

# 分发
scp -r hbase-1.2.6/ hadoop2:/home/hadoop/apps/
scp -r hbase-1.2.6/ hadoop3:/home/hadoop/apps/

6、时间同步：

HBase

集群对于时间的同步要求的比

HDFS

严格，所以，集群启动之前千万记住要进行时间同步，要求相差不要超过 30s：

方法一：

# 配置 hbase-site.xml，三个服务器之间时间差不超过 30s<property><name>hbase.master.maxclockskew</name><value>180000</value><description>Time difference of regionserver from master</description></property>

方法二：

从节点服务器与主节点服务器时间同步
所有节点与网络服务器时间同步

[hadoop@hadoop2 hbase-1.2.6]$ sudo yum install -y ntp

# 与 time.nist.gov 网络服务器同步[hadoop@hadoop2 hbase-1.2.6]$ sudo /usr/sbin/ntpdate time.nist.gov
15 Jun 20:54:27 ntpdate[1780]: step time server 132.163.97.3 offset -28785.531070 sec
            
# 设置定时任务[hadoop@hadoop2 hbase-1.2.6]$ crontab -l
# 服务器时间同步0*/1***/usr/sbin/ntpdate time.nist.gov

参考：http://www.hnbian.cn/posts/f2bc4737.html

7、系统环境变量：

# 所有节点都需要修改[hadoop@hadoop1 bin]$ vim ~/.bashrc

export HBASE_HOME=/home/hadoop/apps/hbase-1.2.6
export PATH=$PATH:$HBASE_HOME/bin
    
source ~/.bashrc

8、启动

hbase

集群：

[hadoop@hadoop1 bin]$ pwd
/home/hadoop/apps/hbase-1.2.6/bin[hadoop@hadoop1 bin]$ start-hbase.sh
starting master, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-master-hadoop1.out
Java HotSpot(TM)64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in8.0
Java HotSpot(TM)64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in8.0
hadoop3: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop3.out
hadoop2: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop2.out
hadoop1: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop1.out
hadoop3: starting master, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-master-hadoop3.out

注意：在哪个集群启动，哪个就是主节点
Master
，启动
hbase
之前一定先启动
zookeeper、hadoop
！！！

9、检查：

[hadoop@hadoop1 bin]$ jps
1668 NameNode
2164 NodeManager
4054 Jps
3590 HMaster
1495 QuorumPeerMain
3719 HRegionServer
1768 DataNode
2061 ResourceManager

[hadoop@hadoop2 hbase-1.2.6]$ jps
1440 QuorumPeerMain
2305 Jps
2149 HRegionServer
1622 NodeManager
1511 DataNode

[hadoop@hadoop3 bin]$ jps
2722 Jps
1682 NodeManager
1436 QuorumPeerMain
1501 DataNode
2654 HRegionServer

若有节点 HRegionServer 进程没起来就手动启动

# 启动 master 进程 
hbase-daemon.sh start master

# 启动HRegionServer进程
hbase-daemon.sh start regionserver

10、

web-ui

：

http://192.168.131.137:16010/

11、测试：

[hadoop@hadoop1 bin]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in[jar:file:/home/hadoop/apps/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in[jar:file:/home/hadoop/apps/hadoop-2.7.5/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type[org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help<RETURN>'forlist of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.6, rUnknown, Mon May 2902:25:32 CDT 2017

hbase(main):001:0>list
TABLE                                                                                                               
0 row(s)in0.6420 seconds

=>[]
hbase(main):002:0> helo 'list'
NoMethodError: undefined method `helo' for#<Object:0x3ab35b9c>

hbase(main):004:0> create 't1',{NAME =>'f1'},{NAME =>'f2'},{NAME =>'f3'}0 row(s)in1.6840 seconds

=> Hbase::Table - t1
hbase(main):005:0>list
TABLE                                                                                                               
t1                                                                                                                  
1 row(s)in0.0290 seconds

=>["t1"]
hbase(main):006:0> desc 't1'
Table t1 is ENABLED                                                                                                 
t1                                                                                                                  
COLUMN FAMILIES DESCRIPTION                                                                                         
{NAME =>'f1', BLOOMFILTER =>'ROW', VERSIONS =>'1', IN_MEMORY =>'false', KEEP_DELETED_CELLS =>'FALSE', DATA_BLOC
K_ENCODING =>'NONE', TTL =>'FOREVER', COMPRESSION =>'NONE', MIN_VERSIONS =>'0', BLOCKCACHE =>'true', BLOCKSIZE 
=>'65536', REPLICATION_SCOPE =>'0'}{NAME =>'f2', BLOOMFILTER =>'ROW', VERSIONS =>'1', IN_MEMORY =>'false', KEEP_DELETED_CELLS =>'FALSE', DATA_BLOC
K_ENCODING =>'NONE', TTL =>'FOREVER', COMPRESSION =>'NONE', MIN_VERSIONS =>'0', BLOCKCACHE =>'true', BLOCKSIZE 
=>'65536', REPLICATION_SCOPE =>'0'}{NAME =>'f3', BLOOMFILTER =>'ROW', VERSIONS =>'1', IN_MEMORY =>'false', KEEP_DELETED_CELLS =>'FALSE', DATA_BLOC
K_ENCODING =>'NONE', TTL =>'FOREVER', COMPRESSION =>'NONE', MIN_VERSIONS =>'0', BLOCKCACHE =>'true', BLOCKSIZE 
=>'65536', REPLICATION_SCOPE =>'0'}3 row(s)in0.3510 seconds

参考文章：

标签：分布式 hadoop 大数据

本文转载自: https://blog.csdn.net/hj1993/article/details/128720122
版权归原作者 风老魔 所有，如有侵权，请联系我们删除。