手把手教你快速在生产环境搭建Doris集群
一、部署规划
组件分布规划
节点node4node5node6node7node8Node9Node10Node11服务FE（follower）
BEFE（follower）
BEFE（follower）
BEFE（observer）
BEBE
BrokerBE
BrokerBEBEIp192.168.22.34192.168.22.35192.168.22.36192.168.22.37192.168.22.20192.168.22.16192.168.22.17192.168.22.18
二、部署准备
安装包准备
根据自己集群的软硬件配置，选择合适的版本进行编译，这里我下载编译doris1.2.2版本源码，推荐使用Docker
https://doris.apache.org/zh-CN/docs/install/source-install/compilation
安装目录准备
1.部署目录规划
为了方便后期升级维护，数据存储，元数据和日志尽量不要放在安装包内。
*FE***部署目录：/opt/module/doris/fe
日志目录：/var/log/doris/fe
元数据目录：/mnt/disk4/doris/metaBE部署目录：/opt/module/doris/be
日志目录：/var/log/doris/be
数据存储目录：/mnt/disk4/doris/storage，/mnt/disk5/doris/storage，/mnt/disk6/doris/storageBroker部署目录：/opt/module/doris/apache_hdfs_broker**
2.数据存储目录创建
在BE上创建数据存储目录，根据规划node4到node11
[root@node4 hadoop]# mkdir -p /mnt/disk4/doris/storage
[root@node4 hadoop]# mkdir -p /mnt/disk5/doris/storage
[root@node4 hadoop]# mkdir -p /mnt/disk6/doris/storage
3.元数据存储目录创建
FE上需要创建元数据存储目录，node4到node7
[root@node4 hadoop]# mkdir -p /mnt/disk4/doris/meta
4.修改数据和元数据文件目录归属
管理Doris时，我们不能使用root用户直接管理，我们使用具有管理权限的hadoop用户进行管理
node4到node11，元数据和数据权限一起修改
[root@node4 hadoop]# chown -R hadoop:hadoop /mnt/disk4/doris
[root@node4 hadoop]# chown -R hadoop:hadoop /mnt/disk5/doris
[root@node4 hadoop]# chown -R hadoop:hadoop /mnt/disk6/doris
5.日志目录创建
BE的日志目录，node4-node11
[root@node4 hadoop]# mkdir -p /var/log/doris/be
FE的日志目录，node4-node7
[root@node4 hadoop]# mkdir -p /var/log/doris/fe
6.修改日志目录权限归属
[root@node4 hadoop]# chown -R hadoop:hadoop /var/log/doris
三、部署安装
配置文件
1.修改FE配置文件
vim /opt/module/doris-1.2.2/fe/conf/fe.conf
日志存储目录
LOG_DIR = /var/log/doris/fe
修改jvm参数，-Xmx调整为16G
JAVA_OPTS="-Xmx16384m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
元数据存储目录
meta_dir = /mnt/disk4/doris/meta
绑定ip
priority_networks = 192.168.22.34/24
2.修改be配置
配置java环境变量
由于从 1.2 版本开始支持 Java UDF 函数，BE 依赖于 Java 环境。所以要预先配置
JAVA_HOME
环境变量，也可以在
start_be.sh
启动脚本第一行添加
export JAVA_HOME=your_java_home_path
来添加环境变量。
vim /opt/module/doris-1.2.2/be/bin/start_be.sh
第一行添加
export JAVA_HOME=/usr/java/jdk
配置be，修改配置文件
vim /opt/module/doris-1.2.2/be/conf/be.conf
日志存储目录
sys_log_dir = /var/log/doris/be
默认的8040端口和hadoop端口冲突了修改为18040
# ports for admin, web, heartbeat service
be_port =9060
webserver_port =18040
heartbeat_service_port =9050
brpc_port =8060
修改绑定ip
priority_networks = 192.168.22.34/24
修改存储目录
路径和硬盘类型间用“，”隔开，多路径用“；”隔开，我这里时机械硬盘medium值为HDD，固态硬盘使用SSD
storage_root_path = /mnt/disk4/doris/storage,medium:HDD;/mnt/disk5/doris/storage,medium:HDD;/mnt/disk6/doris/storage,medium:HDD
3.Broker配置
把hdfs配置文件复制到Broker目录下
mv hdfs-site.xml /opt/module/doris-1.2.2/apache_hdfs_broker/conf/
4.分发与修改配置
4.1.分发
到node4-node11
xsync /opt/module/doris-1.2.2
4.2.修改绑定ip
修改其余节点的绑定ip
node4-node7
vim /opt/module/doris-1.2.2/fe/conf/fe.conf
fe改为对应的绑定ip
...
be修改为对应绑定ip
node8-node11
vim /opt/module/doris-1.2.2/be/conf/be.conf
be修改为对应绑定ip
...
5.添加组件和启动集群
5.1.启动第一个FE
[hadoop@node4 doris]$ fe/bin/start_fe.sh --daemon
查看一下进出是否存在
[hadoop@node4 ~]$ jps |grep PaloFe
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
221813 dorisFE
5.2安装mysql客户端
集群安装mysql客户端
使用客户端连接Fe
mysql -h node4 -P 9030 -uroot
查看fe节点
mysql> show frontends;
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| Name | IP | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| 192.168.22.34_9010_1679021376875 | 192.168.22.34 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 1095009859 | true | true | 144 | 2023-03-17 10:57:15 | true | | doris-1.2.2-rc01-Unknown | Yes |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
第一次连接，没有密码，这里我们修改root密码
SET PASSWORD FOR 'root' = PASSWORD('000000');
5.3.添加其余的FE节点
ALTER SYSTEM ADD FOLLOWER "192.168.22.35:9010";
ALTER SYSTEM ADD FOLLOWER "192.168.22.36:9010";
ALTER SYSTEM ADD OBSERVER "192.168.22.37:9010";
再查看
show frontends
可以看到其他节点FE已经添加上了，只是连不上，因为我们还没有启动
mysql> show frontends;
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+-------+-------+-------------------+---------------------+----------+--------------------------------------------------------------+--------------------------+------------------+
| Name | IP | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+-------+-------+-------------------+---------------------+----------+--------------------------------------------------------------+--------------------------+------------------+
| 192.168.22.36_9010_1679022090994 | 192.168.22.36 | 9010 | 8030 | 0 | 0 | FOLLOWER | false | 1095009859 | false | false | 0 | NULL | true | java.net.ConnectException: 拒绝连接 (Connection refused) | NULL | No |
| 192.168.22.34_9010_1679021376875 | 192.168.22.34 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 1095009859 | true | true | 232 | 2023-03-17 11:01:57 | true | | doris-1.2.2-rc01-Unknown | Yes |
| 192.168.22.35_9010_1679022090967 | 192.168.22.35 | 9010 | 8030 | 0 | 0 | FOLLOWER | false | 1095009859 | false | false | 0 | NULL | true | java.net.ConnectException: 拒绝连接 (Connection refused) | NULL | No |
| 192.168.22.37_9010_1679022093117 | 192.168.22.37 | 9010 | 8030 | 0 | 0 | OBSERVER | false | 1095009859 | false | false | 0 | NULL | false | java.net.ConnectException: 拒绝连接 (Connection refused) | NULL | No |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+-------+-------+-------------------+---------------------+----------+--------------------------------------------------------------+--------------------------+------------------+
4 rows in set (0.02 sec)
5.4.启动其他FE
注意：第一次启动需指定–helper参数，后续再启动无需指定此参数
node5-node7
分别进入对应节点的fe目录
cd /opt/module/doris-1.2.2/fe
分别执行如下命令
bin/start_fe.sh --helper 192.168.22.34:9010 --daemon
分别查看FE进程
[hadoop@node7 fe]$ jps | grep PaloFe
12644 PaloFe
第三次使用mysql客户端查看
show frontends
结果如下
mysql> show frontends;
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| Name | IP | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
| 192.168.22.36_9010_1679022090994 | 192.168.22.36 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | false | 1095009859 | true | true | 357 | 2023-03-17 11:09:03 | true | | doris-1.2.2-rc01-Unknown | No |
| 192.168.22.34_9010_1679021376875 | 192.168.22.34 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 1095009859 | true | true | 359 | 2023-03-17 11:09:03 | true | | doris-1.2.2-rc01-Unknown | Yes |
| 192.168.22.35_9010_1679022090967 | 192.168.22.35 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | false | 1095009859 | true | true | 357 | 2023-03-17 11:09:03 | true | | doris-1.2.2-rc01-Unknown | No |
| 192.168.22.37_9010_1679022093117 | 192.168.22.37 | 9010 | 8030 | 9030 | 9020 | OBSERVER | false | 1095009859 | true | true | 357 | 2023-03-17 11:09:03 | false | | doris-1.2.2-rc01-Unknown | No |
+-----------------------------------+----------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+--------------------------+------------------+
注意看4个FE全部处于激活状态
5.5.添加BE
mysql客户端执行对应的如下语句
ALTER SYSTEM ADD BACKEND "192.168.22.34:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.35:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.36:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.37:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.20:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.16:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.17:9050";
ALTER SYSTEM ADD BACKEND "192.168.22.18:9050";
6.6.启动其他BE
node4-node11
进入BE目录
cd /opt/module/doris/be
启动
/be/bin/start_be.sh --daemon
提示(若没有如下提示，跳过)
Please set vm.max_map_count to be 2000000 under root using 'sysctl -w vm.max_map_count=2000000'.
根据提示，分别在be节点执行
sysctl -w vm.max_map_count=2000000
再在各个be节点，启动be。
查看状况
mysql客户端
SHOW backends;
mysql> SHOW backends;
+-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| BackendId | Cluster | IP | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | RemoteUsedCapacity | Tag | ErrMsg | Version | Status | HeartbeatFailureCounter | NodeRole |
+-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
| 11006 | default_cluster | 192.168.22.16 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:14:29 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 10.567 TB | 10.911 TB | 3.15 % | 3.24 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:37","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11007 | default_cluster | 192.168.22.17 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:14:29 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 10.575 TB | 10.911 TB | 3.08 % | 3.11 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11008 | default_cluster | 192.168.22.18 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:14:30 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 10.562 TB | 10.911 TB | 3.20 % | 3.24 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11001 | default_cluster | 192.168.22.34 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:12 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 7.731 TB | 10.744 TB | 28.05 % | 28.51 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:25","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11002 | default_cluster | 192.168.22.35 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:13 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 7.832 TB | 10.744 TB | 27.11 % | 27.34 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11003 | default_cluster | 192.168.22.36 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:12 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 8.204 TB | 10.744 TB | 23.64 % | 24.17 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:41","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11004 | default_cluster | 192.168.22.37 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:14 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 8.424 TB | 10.744 TB | 21.59 % | 23.50 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:33","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
| 11005 | default_cluster | 192.168.22.20 | 9050 | 9060 | 18040 | 8060 | 2023-03-17 13:09:16 | 2023-03-17 13:14:52 | true | false | false | 0 | 0.000 | 7.340 TB | 10.744 TB | 31.68 % | 31.77 % | 0.000 | {"location" : "default"} | | doris-1.2.2-rc01-Unknown | {"lastSuccessReportTabletsTime":"2023-03-17 13:14:35","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | 0 | mix |
+-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------+--------------------------+--------+--------------------------+-------------------------------------------------------------------------------------------------------------------------------+-------------------------+----------+
查看进程
[hadoop@node4 be]$ ps -ef |grep doris_be
hadoop 522601017:23 pts/2 00:00:08 /opt/module/doris/be/lib/doris_be
6.7.添加Broker
mySql
ALTER SYSTEM ADD BROKER broker8 "192.168.22.38:8000";
ALTER SYSTEM ADD BROKER broker9 "192.168.22.39:8000";
6.8.启动broker
node8和node9上启动
[hadoop@node8 doris]$ cd apache_hdfs_broker/
[hadoop@node8 apache_hdfs_broker]$ bin/start_broker.sh --daemon
查看进程
[hadoop@node8 apache_hdfs_broker]$ jps |grep BrokerBootstrap
37003 BrokerBootstrap
6.9.编写集群启停脚本
群起脚本
#!/bin/bash# use-method: starrocks.sh start|stop|restartfe_hosts=(node4 node5 node6 node7)be_hosts=(node4 node5 node6 node7 node8 node9 node10 node11)broker_hosts=(node8 kafka1)case$1in"start"){foriin${fe_hosts[*]}doecho"=================== start $i's fe service ================"ssh$i"source /etc/profile.d/my_env.sh ;cd /opt/module/starRocks;./fe/bin/start_fe.sh --daemon"doneforiin${be_hosts[*]}doecho"=================== start $i's be service ================"ssh$i"source /etc/profile.d/my_env.sh ;cd /opt/module/starRocks;./be/bin/start_be.sh --daemon"doneforiin${broker_hosts[*]}doecho"=================== start $i's broker service ================"# ssh $i "source /etc/profile.d/my_env.sh ;cd /opt/module/starRocks;./apache_hdfs_broker/bin/start_broker.sh --daemon"done};;"stop"){foriin${broker_hosts[*]}doecho"=================== stop $i's broker service ================"ssh$i"/opt/module/starRocks/apache_hdfs_broker/bin/stop_broker.sh"doneforiin${be_hosts[*]}doecho"=================== stop $i's be service ================"ssh$i"/opt/module/starRocks/be/bin/stop_be.sh"doneforiin${fe_hosts[*]}doecho"=================== stop $i's fe service ================"ssh$i"/opt/module/starRocks/fe/bin/stop_fe.sh"done};;"status"){foriin${fe_hosts[*]}doecho"=================== status $i's fe service ================"ssh$i"jps | grep StarRocksFE"doneforiin${be_hosts[*]}doecho"=================== status $i's be service ================"ssh$i"ps -ef | grep starrocks_be | grep -v 'grep'"doneforiin${broker_hosts[*]}doecho"=================== status $i's broker service ================"ssh$i"jps | grep BrokerBootstrap | grep -v ‘JAVA_TOOL_OPTIONS’"done};;"restart")
starrocks-cluster.sh stop
sleep2
starrocks-cluster.sh start
;;
*)echo"Parameter ERROR!!! starrocks.sh start|stop|restart";;esac
添加脚本执行权限
sudo +x doris-cluster.sh
6.10.web界面
be的web界面
http://node4.bigdata.59wanmei.com:18040/
http://node5.bigdata.59wanmei.com:18040/
http://node6.bigdata.59wanmei.com:18040/
http://node7.bigdata.59wanmei.com:18040/
。。。
fe的web界面
http://node4.bigdata.59wanmei.com:8030/home
http://node5.bigdata.59wanmei.com:8030/home
http://node6.bigdata.59wanmei.com:8030/home
http://node7.bigdata.59wanmei.com:8030/home
四、账号权限管理
1.添加一个账号
mysql -h node4 -P 9030 -uroot -p'000000'
mysql客户端执行
set password=password('000000');
创建数据库
createdatabase test_db;
创建一个普通用户
CREATEUSER test IDENTIFIED BY'test000000';
给当前用户bus_log库的一个只读权限
GRANT SELECT_PRIV ON test_db.*TO test;
好了，集群搭建完成。
本文转载自: https://blog.csdn.net/weixin_52918377/article/details/129622413
版权归原作者 薛定谔的猫不吃猫粮 所有， 如有侵权，请联系我们删除。
版权归原作者 薛定谔的猫不吃猫粮 所有， 如有侵权，请联系我们删除。