0


启动hive时去掉没有用的日志信息

背景:自己安装了hive,但是输入hive命令,显示的日志信息太多了,都是没啥用的信息,实在影响自己的使用体验

启动hive显示如下信息
[root@master sbin]# hive
2023-03-1610:24:38,960INFO[main] conf.HiveConf: Found configuration file file:/root/soft/hive/apache-hive-3.1.3-bin/conf/hive-site.xml
Hive Session ID= 7e094e16-f7a5-4c64-9ad4-153c5ea28fe5
2023-03-1610:24:39,576INFO[main] SessionState: Hive Session ID= 7e094e16-f7a5-4c64-9ad4-153c5ea28fe5

Logging initialized using configuration injar:file:/root/soft/hive/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async:true2023-03-1610:24:39,596INFO[main] SessionState: 
Logging initialized using configuration injar:file:/root/soft/hive/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async:true2023-03-1610:24:40,037INFO[main] session.SessionState: Created HDFSdirectory:/tmp/hive/hdfs
2023-03-1610:24:40,039INFO[main] session.SessionState: Created HDFSdirectory:/tmp/hive/hdfs/7e094e16-f7a5-4c64-9ad4-153c5ea28fe5
2023-03-1610:24:40,048INFO[main] session.SessionState: Created local directory:/tmp/root/7e094e16-f7a5-4c64-9ad4-153c5ea28fe5
2023-03-1610:24:40,051INFO[main] session.SessionState: Created HDFSdirectory:/tmp/hive/hdfs/7e094e16-f7a5-4c64-9ad4-153c5ea28fe5/_tmp_space.db
2023-03-1610:24:40,057INFO[main] conf.HiveConf: Using the default value passed infor log id: 7e094e16-f7a5-4c64-9ad4-153c5ea28fe5
2023-03-1610:24:40,057INFO[main] session.SessionState: Updating thread name to 7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main
2023-03-1610:24:40,383INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.HiveMetaStore:0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore
2023-03-1610:24:40,496WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null. Setting it to value: ignored
2023-03-1610:24:40,500INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.ObjectStore: ObjectStore, initialize called
2023-03-1610:24:40,500INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.MetastoreConf: Found configuration file file:/root/soft/hive/apache-hive-3.1.3-bin/conf/hive-site.xml
2023-03-1610:24:40,501INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.MetastoreConf: Unable to find config file hivemetastore-site.xml
2023-03-1610:24:40,501INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.MetastoreConf: Found configuration file null2023-03-1610:24:40,501INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.MetastoreConf: Unable to find config file metastore-site.xml
2023-03-1610:24:40,501INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.MetastoreConf: Found configuration file null2023-03-1610:24:40,575INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
2023-03-1610:24:40,656INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] hikari.HikariDataSource: HikariPool-1- Starting...2023-03-1610:24:40,889INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] hikari.HikariDataSource: HikariPool-1- Start completed.2023-03-1610:24:40,908INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] hikari.HikariDataSource: HikariPool-2- Starting...2023-03-1610:24:40,920INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] hikari.HikariDataSource: HikariPool-2- Start completed.2023-03-1610:24:40,956INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"2023-03-1610:24:41,000INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL2023-03-1610:24:41,000INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.ObjectStore: Initialized ObjectStore
2023-03-1610:24:41,093WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,093WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,093WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,093WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,094WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,094WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,676WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,676WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,676WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,676WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,677WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:41,677WARN[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] DataNucleus.MetaData: Metadata has jdbc-type ofnull yet this is not valid. Ignored
2023-03-1610:24:42,546INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.HiveMetaStore: Added admin role in metastore
2023-03-1610:24:42,547INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.HiveMetaStore: Added public role in metastore
2023-03-1610:24:42,556INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.HiveMetaStore: No user is added in admin role, since config is empty
2023-03-1610:24:42,766INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=classorg.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=hdfs(auth:SIMPLE) retries=1 delay=1 lifetime=02023-03-1610:24:42,775INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.HiveMetaStore:0: get_all_functions
2023-03-1610:24:42,776INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] HiveMetaStore.audit: ugi=hdfs ip=unknown-ip-addr      cmd=get_all_functions
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine(i.e. spark, tez) or using Hive 1.X releases.2023-03-1610:24:42,791INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] CliDriver: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine(i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID= 87f816df-986d-41f0-9198-6f2899b10286
2023-03-1610:24:42,791INFO[pool-10-thread-1] SessionState: Hive Session ID= 87f816df-986d-41f0-9198-6f2899b10286
2023-03-1610:24:42,799INFO[pool-10-thread-1] session.SessionState: Created HDFSdirectory:/tmp/hive/hdfs/87f816df-986d-41f0-9198-6f2899b10286
2023-03-1610:24:42,800INFO[pool-10-thread-1] session.SessionState: Created local directory:/tmp/root/87f816df-986d-41f0-9198-6f2899b10286
2023-03-1610:24:42,802INFO[pool-10-thread-1] session.SessionState: Created HDFSdirectory:/tmp/hive/hdfs/87f816df-986d-41f0-9198-6f2899b10286/_tmp_space.db
2023-03-1610:24:42,802INFO[pool-10-thread-1] metastore.HiveMetaStore:1: get_databases: @hive#
2023-03-1610:24:42,802INFO[pool-10-thread-1] HiveMetaStore.audit: ugi=hdfs  ip=unknown-ip-addr      cmd=get_databases: @hive#
2023-03-1610:24:42,803INFO[pool-10-thread-1] metastore.HiveMetaStore:1: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore
2023-03-1610:24:42,803INFO[pool-10-thread-1] metastore.ObjectStore: ObjectStore, initialize called
2023-03-1610:24:42,814INFO[pool-10-thread-1] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL2023-03-1610:24:42,814INFO[pool-10-thread-1] metastore.ObjectStore: Initialized ObjectStore
2023-03-1610:24:42,819INFO[pool-10-thread-1] metastore.HiveMetaStore:1: get_tables_by_type: db=@hive#default pat=.*,type=MATERIALIZED_VIEW2023-03-1610:24:42,819INFO[pool-10-thread-1] HiveMetaStore.audit: ugi=hdfs  ip=unknown-ip-addr      cmd=get_tables_by_type: db=@hive#default pat=.*,type=MATERIALIZED_VIEW2023-03-1610:24:42,822INFO[pool-10-thread-1] metastore.HiveMetaStore:1: get_multi_table : db=default tbls=2023-03-1610:24:42,822INFO[pool-10-thread-1] HiveMetaStore.audit: ugi=hdfs  ip=unknown-ip-addr      cmd=get_multi_table : db=default tbls=2023-03-1610:24:42,822INFO[pool-10-thread-1] metadata.HiveMaterializedViewsRegistry: Materialized views registry has been initialized
输入命令,显示如下信息,show databases;
hive> show databases;2023-03-1610:28:41,922INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.HiveConf: Using the default value passed infor log id: 7e094e16-f7a5-4c64-9ad4-153c5ea28fe5
2023-03-1610:28:41,995INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Compiling command(queryId=root_20230316102841_e9d4cc33-8d34-4912-ba9f-83c8b873d1db): show databases
2023-03-1610:28:42,239INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-1610:28:42,246INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Semantic Analysis Completed(retrial =false)2023-03-1610:28:42,262INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Returning Hive schema:Schema(fieldSchemas:[FieldSchema(name:database_name,type:string,comment:from deserializer)],properties:null)2023-03-1610:28:42,291INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] exec.ListSinkOperator: Initializing operator LIST_SINK[0]2023-03-1610:28:42,294INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Completed compiling command(queryId=root_20230316102841_e9d4cc33-8d34-4912-ba9f-83c8b873d1db); Time taken:0.318 seconds
2023-03-1610:28:42,294INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] reexec.ReExecDriver: Execution #1of query
2023-03-1610:28:42,295INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-1610:28:42,295INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Executing command(queryId=root_20230316102841_e9d4cc33-8d34-4912-ba9f-83c8b873d1db): show databases
2023-03-1610:28:42,478INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Starting task [Stage-0:DDL]in serial mode
2023-03-1610:28:42,479INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] metastore.HiveMetaStore:0: get_databases: @hive#
2023-03-1610:28:42,480INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] HiveMetaStore.audit: ugi=hdfs ip=unknown-ip-addr      cmd=get_databases: @hive#
2023-03-1610:28:42,483INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] exec.DDLTask: results :12023-03-1610:28:42,488INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Completed executing command(queryId=root_20230316102841_e9d4cc33-8d34-4912-ba9f-83c8b873d1db); Time taken:0.193 seconds
OK2023-03-1610:28:42,488INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver:OK2023-03-1610:28:42,488INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2023-03-1610:28:42,493INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
2023-03-1610:28:42,524INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] mapred.FileInputFormat: Total input files to process :12023-03-1610:28:42,539INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] exec.ListSinkOperator:RECORDS_OUT_INTERMEDIATE:0,RECORDS_OUT_OPERATOR_LIST_SINK_0:1,default
Time taken:0.514 seconds,Fetched:1row(s)2023-03-1610:28:42,542INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] CliDriver: Time taken:0.514 seconds,Fetched:1row(s)2023-03-1610:28:42,542INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] conf.HiveConf: Using the default value passed infor log id: 7e094e16-f7a5-4c64-9ad4-153c5ea28fe5
2023-03-1610:28:42,542INFO[7e094e16-f7a5-4c64-9ad4-153c5ea28fe5 main] session.SessionState: Resetting thread name to  main
解决方法:在你安装hive的conf目录下,新建log4j.properties日志的配置文件即可

日志内容如下;
log4j.rootLogger:这个是设置日志级别的,当设置成不同的日志级别时,展示的日志输出是不一样的

log4j.rootLogger=WARN,CA           
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%-4r [%t]%-5p %c %x -%m%n

在这里插入图片描述
当日志级别为WARN时,打印的还是不少日志信息

Hive Session ID= a3330e2f-5e32-4da2-9ab0-502e06798508

Logging initialized using configuration injar:file:/root/soft/hive/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async:true0[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  org.apache.hadoop.hive.metastore.ObjectStore  - datanucleus.autoStartMechanismMode is set to unsupported value null. Setting it to value: ignored
588[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
588[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
589[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
589[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
589[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
589[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
1266[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
1266[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
1267[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
1267[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
1267[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
1267[a3330e2f-5e32-4da2-9ab0-502e06798508 main]WARN  DataNucleus.MetaData  - Metadata has jdbc-type ofnull yet this is not valid. Ignored
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine(i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID= 9b00f550-1e05-4328-a9ab-e6e8b3abaa0d
当将日志级别设置成ERROR时,看着是不是特别清爽了
[root@master conf]# hive
Hive Session ID= 98032ca8-bfb5-4ae6-bba7-93ec57d6c371

Logging initialized using configuration injar:file:/root/soft/hive/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async:true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine(i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID= c3f15910-9d83-4916-ba25-2c8f9138e3da

非常nice,😄
在这里插入图片描述

标签: hive hadoop 大数据

本文转载自: https://blog.csdn.net/qq_38220334/article/details/130603666
版权归原作者 呼伦贝尔-钢蛋儿 所有, 如有侵权,请联系我们删除。

“启动hive时去掉没有用的日志信息”的评论:

还没有评论