Hive本地开发/学习环境配置

前提

hive依赖hadoop的相关组件，需要启动Hadoop的相关组件。

Hive 版本：3.1.3
Hadoop版本：3.3.4

hive-env.sh

exportHADOOP_HOME=$HADOOP_HOMEexportHIVE_CONF_DIR=/usr/local/Cellar/hive/3.1.3/libexec/conf
exportHIVE_AUX_JARS_PATH=/usr/local/Cellar/hive/3.1.3/libexec/lib

hive-site.xml

<configuration><property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value><description>JDBC connect string for a JDBC metastore</description></property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for a JDBC metastore</description></property><property><name>javax.jdo.option.ConnectionUserName</name><value>root</value><description>username to use against metastore database</description></property><property><name>javax.jdo.option.ConnectionPassword</name><value>root</value><description>password to use against metastore database</description></property><property><name>hive.cli.print.header</name><value>true</value></property><property><name>hive.cli.print.current.db</name><value>true</value></property><property><name>hive.metastore.schema.verification</name><value>false</value></property><!-- hiveserver2运行绑定的host --><property><name>hive.server2.thrift.bind.host</name><value>localhost</value></property><!-- 远程模式部署metastore 的地址 --><property><name>hive.metastore.uris</name><value>thrift://localhost:9083</value></property><!-- 配置hiveserver2服务启动问题（不配置，则hiveserver无法监听服务，会导致beeline客户端无法连接） --><property><name>hive.metastore.event.db.notification.api.auth</name><value>false</value></property></configuration>

hive初始化

需要先启动Hadoop的相关组件服务。

schematool -dbType mysql -initSchema --verbose

注意：一定要先

-dbType mysql

然后才

-initSchema

，否则会出现连接MySQL的错误，

Metastore connection URL:     jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&useSSL=falseMetastoreConnectionDriver:com.mysql.jdbc.DriverMetastore connection User:     root
org.apache.hadoop.hive.metastore.HiveMetaException:Failedtoget schema version.
Underlying cause:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:Communications link failure

The last packet sent successfully tothe server was 0 milliseconds ago. The driver has not received any packets from the server.
SQL Error code:0org.apache.hadoop.hive.metastore.HiveMetaException:Failedtoget schema version.
    at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:94)
    at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169)
    at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475)
    at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581)
    at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)
    at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:236)Caused by:com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:Communications link failure

The last packet sent successfully tothe server was 0 milliseconds ago. The driver has not received any packets from the server.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(NativeMethod)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:403)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:335)
    at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2187)
    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2220)
    at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2015)
    at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:768)
    at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(NativeMethod)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:403)
    at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:385)
    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:323)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:247)
    at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)...11 more
Caused by:java.net.ConnectException:Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(NativeMethod)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:606)
    at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:299)...26 more
*** schemaTool failed ***

metastore服务启动

进入hive的bin目录，执行命令

hive --service metastore 

,或者使用nohup命令进行后台启动，

nohup hive --service metastore &

，这样就启动了metastore服务，可以通过jps命令查看，是否存在RunJar的进程，存在则说明启动成功，再通过hive客户端连接即可。

hiveserver2服务启动

该服务依赖于metastore服务，所以，在启动hiveserver2之前，一定确保已经启动了metastore服务，这时，可以通过

hive --service hiveserver2

命令启动服务，当然同样可以通过nohup命令进行后台启动，

nohup hive --service hiveserver2 &

，启动后，通过jps命令查看服务进程，此时会多出一个RunJar的进程，即为hiverserver2的服务。

通过beeline客户端进行连接，此时还没有真正连接上。

还需要在当前客户端窗口输入hiveserver2的连接信息，

! connect jdbc:hive2://localhost:10000

，hive2默认的端口为10000，此时，会提示让你输入用户名名，然后再输入密码。

Error:Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default:java.net.ConnectException refused

出现上述错误

需要在hive-site。添加如下配置

<property><name>hive.metastore.event.db.notification.api.auth</name><value>false</value></property>

然后重新启动相关服务，再使用beeline客户端连接，出现了如下错误：

beeline> !connect jdbc:hive2://localhost:10000

Connecting to jdbc:hive2://localhost:10000

Enter username for jdbc:hive2://localhost:10000: yuxiao

Enter password for jdbc:hive2://localhost:10000:

23/05/01 18:14:01 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000

Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: yuxiao is not allowed to impersonate yuxiao (state=08S01,code=0)

这个问题是由于不允许你当前登录的用户访问hadoop，所以需要调整hadoop的core-site.xml的配置，添加相关proxuser的访问即可，

<property><name>hadoop.proxyuser.用户名.hosts</name><value>*</value></property><property><name>hadoop.proxyuser.用户名.groups</name><value>*</value></property>

将配置文件的用户修改上hive不允许访问的用户名，配置文件修改后，重新启动hadoop的dfs服务，

再使用beeline客户端访问，用户名输入之前提示不允许访问的用户名，密码直接回车即可，这样就可以正常访问hive了，如下图所示：

查看数据库：

hive webui配置

hiveserver2服务具有可视化界面，查看相关信息，需要在hive-site.xml文件配置webui，如下所示：

<property><name>hive.server2.webui.host</name><value>127.0.0.1</value></property><property><name>hive.server2.webui.port</name><value>10002</value></property>

在启动hiveserver2服务后，访问10002端口查看webui，如下：