Apache 官方 Release 版本请前往Downloads | Apache Doris下载。
这里下载的是百度Palo预编译版本,不需要编译源码,直接安装就行
Palo文档PALO - 预编译版本下载 | Doris (baidu.com)
官方建议 10 ~ 100 台左右的机器,来充分发挥 Doris 的性能(其中 3 台部署 FE(HA),剩余的部署 BE),最少4台机器(一台 FE,三台 BE),我们部署4台简易版。
1.下载。
0.14.13.1 及之后的版本需要运行环境的 CPU 支持 avx2 指令,你可以通过 cat /proc/cpuinfo 查看是否支持,这里我们是是不支持avx2指令。
0.15.1-rc09
是支持avx2指令,
0.15.1-rc09-no-avx2
是不支持avx2指令,我下载的是
0.15.1-rc09-no-avx2。
2.安装前系统软件准备(所有机器)
1)Linux 文件系统
在这里,我们建议使用 ext4 文件系统。安装操作系统时,请选择 ext4 文件系统
2)时钟同步
Doris 的元数据要求时间精度小于 5000ms,因此集群中的所有机器都需要同步时钟,以避免因时钟问题导致元数据不一致而导致的服务异常。
Linux 时间同步 Chrony
3)关闭交换分区
Linux交换分区会给Doris带来严重的性能问题,需要在安装前禁用交换分区
1、不重启电脑,禁用启用swap,立刻生效
禁用命令 sudo swapoff -a
启用命令 sudo swapon -a
查看交换分区的状态 sudo free -m
2、重新启动电脑,永久禁用Swap
把根目录文件系统设为可读写
sudo mount -n -o remount,rw /
用vi修改/etc/fstab文件,在swap分区这行前加 # 禁用掉,保存退出
vi /etc/fstab
重新启动电脑,使用free -m查看分区状态
reboot
sudo free -m
4)把limits调到最大
1、查看打开文件的限制数 ulimit -a
显示文件的打开数项为open files (-n) 1024修改
2、修改
临时生效:ulimit -n 65536 (不需要设,直接设置永久)
永久生效:
vim /etc/security/limits.conf添加
- soft nofile 65536
- hard nofile 65536
vim /etc/sysctl.conf中添加fs.file-max=655350
/sbin/sysctl -p 使立即生效
cat /proc/sys/fs/file-max 查看是否设置成功
5)安装gcc,jdk
SoftVersionJava1.8 及以上GCC4.8.2 及以上
6) 在所有BE节点上安装ODBC
Centos7 安装ODBC访问 Mysql
3.部署Doris
1)上传至服务器
/opt/software
目录,每台服务器都需要上传;解压。
[root@slave1 software]# tar -zxvf PALO-0.15.1-rc09-no-avx2-binary
第一台机器把
fe
挪到
/home/doris/
目录下去,前提需要创建该目录
root@slave1 PALO-0.15.1-rc09-no-avx2-binary]# mv fe /home/doris/
剩下三台机器把be挪到
/home/doris/
目录下去
root@slave2 PALO-0.15.1-rc09-no-avx2-binary]# mv be /home/doris/
root@slave3 PALO-0.15.1-rc09-no-avx2-binary]# mv be /home/doris/
root@slave4 PALO-0.15.1-rc09-no-avx2-binary]# mv be /home/doris/
2)进入
/home/doris/fe/conf
,
vim be.conf
,需要修改配置
下面我的配置做个参考:
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#####################################################################
## The uppercase properties are read and exported by bin/start_fe.sh.
## To see all Frontend configurations,
## see fe/src/org/apache/doris/common/Config.java
#####################################################################
# the output dir of stderr and stdout
LOG_DIR = ${DORIS_HOME}/log
DATE = `date +%Y%m%d-%H%M%S`
JAVA_OPTS="-Xmx32768m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"
# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Xmx32768m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$DATE:time"
##
## the lowercase properties are read by main program.
##
# INFO, WARN, ERROR, FATAL
sys_log_level = INFO
# store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
# meta_dir = ${DORIS_HOME}/doris-meta
meta_dir = /home/doris/fe/doris-meta
http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
mysql_service_nio_enabled = true
enable_http_server_v2 = true
# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.0.111/24
# Advanced configurations
# log_roll_size_mb = 1024
# sys_log_dir = ${DORIS_HOME}/log
# sys_log_roll_num = 10
# sys_log_verbose_modules = org.apache.doris
# audit_log_dir = ${DORIS_HOME}/log
# audit_log_modules = slow_query, query
# audit_log_roll_num = 10
# meta_delay_toleration_second = 10
# qe_max_connection = 1024
# max_conn_per_user = 100
# qe_query_timeout_second = 300
# qe_slow_log_ms = 5000
dynamic_partition_enable = true
enable_http_server_v2=true
3)启动fe。
启动和关闭fe的命令是
sh start_fe.sh --daemon
sh stop_fe.sh --daemon
通过MySQL客户端连接到FE,这表明FE已成功启动
mysql官网:
MySQL :: MySQL Downloads
mysql-5.7.36-linux-glibc2.12-x86_64.tar.gz
[root@slave1 software]# tar -zxvf mysql-5.7.36-linux-glibc2.12-x86_64.tar.gz
[root@slave1 software]# cd mysql-5.7.36-linux-glibc2.12-x86_64.tar.gz /bin
[root@slave1 bin]# ./mysql -h 192.168.0.111 -P 9030 -uroot
查看fe节点的状态,主要关注
Role
角色、
IsMaster
是否主节点、
alive
节点存活状态 三列是否正常
mysql> show proc '/frontends';
4)安装BE
进入
/home/doris/be/conf
目录下,修改配置
下面我的配置做个参考:
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
PPROF_TMPDIR="$DORIS_HOME/log/"
# INFO, WARNING, ERROR, FATAL
sys_log_level = INFO
# ports for admin, web, heartbeat service
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
priority_networks = 192.168.0.222/24
#storage_root_path = /home/doris/doris-data1.hdd,50; /home/doris/doris-data2.ssd,50; /home/doris/doris-data3.ssd,50
max_tablet_version_num = 5000
max_compaction_threads = 16
compaction_task_num_per_disk = 8
cumulative_size_based_promotion_size_mbytes = 2048
row_step_for_compaction_merge_log = 1
total_permits_for_compaction_score = 50000
# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
# data root path, separate by ';'
# you can specify the storage medium of each root path, HDD or SSD
# you can add capacity limit at the end of each root path, seperate by ','
# eg:
# storage_root_path = /home/disk1/doris.HDD,50;/home/disk2/doris.SSD,1;/home/disk2/doris
# /home/disk1/doris.HDD, capacity limit is 50GB, HDD;
# /home/disk2/doris.SSD, capacity limit is 1GB, SSD;
# /home/disk2/doris, capacity limit is disk capacity, HDD(default)
#
# you also can specify the properties by setting '<property>:<value>', seperate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
storage_root_path = /home/doris/be/storage
3台机器都按照以上内容进行配置,后者
be.conf
改一下
ip
地址就行。
启动所有
be
节点
cd /home/doris/be/bin
sh start_be.sh --daemon
5)进入
fe
节点,用mysql连接进入sql命令行模式,挨个添加be节点
[root@slave1 software]# cd mysql-5.7.36-linux-glibc2.12-x86_64.tar.gz /bin
[root@slave1 bin]# ./mysql -h 192.168.0.111 -P 9030 -uroot
mysql> ALTER SYSTEM ADD BACKEND "192.168.0.222:9050";
mysql> ALTER SYSTEM ADD BACKEND "192.168.0.333:9050";
mysql> ALTER SYSTEM ADD BACKEND "192.168.0.444:9050";
查看
be
节点的状态,主要关注
alive
节点存活状态 是否正常。
mysql> show proc '/backends';
进入网页端
http://192.168.0.111:8030
可以去fe节点改密码,没有的话,直接进。
也可以用navicat连接,或DBeaver
连接MySQL方式一样,端口9030;
版权归原作者 红烧佩奇真好吃 所有, 如有侵权,请联系我们删除。