0


flink消费kafka报错

完整报错:

ERROR coordinator.SourceCoordinatorContext: Exception while handling result from async call in SourceCoordinator-Source: source. Triggering job failover.
org.apache.flink.util.FlinkRuntimeException: Failed to list subscribed topic partitions due to
at org.apache.flink.connector.kafka.source.enumerator.KafkaSourceEnumerator.checkPartitionChanges(KafkaSourceEnumerator.java:234)
at org.apache.flink.runtime.source.coordinator.ExecutorNotifier.lambda$null$1(ExecutorNotifier.java:83)
at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoSuchMethodError: org.apache.kafka.clients.admin.DescribeTopicsResult.allTopicNames()Lorg/apache/kafka/common/KafkaFuture;
at org.apache.flink.connector.kafka.source.enumerator.subscriber.KafkaSubscriberUtils.getTopicMetadata(KafkaSubscriberUtils.java:44)
at org.apache.flink.connector.kafka.source.enumerator.subscriber.TopicListSubscriber.getSubscribedTopicPartitions(TopicListSubscriber.java:52)
at org.apache.flink.connector.kafka.source.enumerator.KafkaSourceEnumerator.getSubscribedTopicPartitions(KafkaSourceEnumerator.java:219)
at org.apache.flink.runtime.source.coordinator.ExecutorNotifier.lambda$notifyReadyAsync$2(ExecutorNotifier.java:80)
...

依赖问题:

查看是否同时含有

  1. <dependency>
  2. <groupId>org.apache.kafka</groupId>
  3. <artifactId>kafka-clients</artifactId>
  4. <version>3.0.0</version>
  5. </dependency>
  1. <dependency>
  2. <groupId>org.apache.flink</groupId>
  3. <artifactId>flink-connector-kafka</artifactId>
  4. <version>${flink.version}</version>
  5. </dependency>

版本依赖问题,删除kafka-clients这个有依赖即可,只保留

  1. <dependency>
  2. <groupId>org.apache.flink</groupId>
  3. <artifactId>flink-connector-kafka</artifactId>
  4. <version>${flink.version}</version>
  5. </dependency>

或新建一个项目导入以下依赖(注意版本问题,修改为自己的版本):

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <project xmlns="http://maven.apache.org/POM/4.0.0"
  3. xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  4. xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  5. <modelVersion>4.0.0</modelVersion>
  6. <groupId>org.example</groupId>
  7. <artifactId>flink_test</artifactId>
  8. <version>1.0-SNAPSHOT</version>
  9. <properties>
  10. <maven.compiler.source>8</maven.compiler.source>
  11. <maven.compiler.target>8</maven.compiler.target>
  12. <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  13. <flink.version>1.17.2</flink.version>
  14. </properties>
  15. <dependencies>
  16. <dependency>
  17. <groupId>org.apache.flink</groupId>
  18. <artifactId>flink-streaming-java</artifactId>
  19. <version>${flink.version}</version>
  20. <scope>provided</scope>
  21. </dependency>
  22. <dependency>
  23. <groupId>org.apache.flink</groupId>
  24. <artifactId>flink-clients</artifactId>
  25. <version>${flink.version}</version>
  26. <scope>provided</scope>
  27. </dependency>
  28. <dependency>
  29. <groupId>org.apache.flink</groupId>
  30. <artifactId>flink-runtime-web</artifactId>
  31. <version>${flink.version}</version>
  32. <scope>provided</scope>
  33. </dependency>
  34. <dependency>
  35. <groupId>org.apache.flink</groupId>
  36. <artifactId>flink-connector-kafka</artifactId>
  37. <version>${flink.version}</version>
  38. </dependency>
  39. <dependency>
  40. <groupId>org.apache.flink</groupId>
  41. <artifactId>flink-json</artifactId>
  42. <version>1.13.6</version>
  43. <scope>provided</scope>
  44. </dependency>
  45. <dependency>
  46. <groupId>org.apache.flink</groupId>
  47. <artifactId>flink-table-api-java-bridge</artifactId>
  48. <version>${flink.version}</version>
  49. <scope>provided</scope>
  50. </dependency>
  51. <dependency>
  52. <groupId>org.apache.flink</groupId>
  53. <artifactId>flink-table-planner-loader</artifactId>
  54. <version>${flink.version}</version>
  55. <scope>provided</scope>
  56. </dependency>
  57. <dependency>
  58. <groupId>org.apache.flink</groupId>
  59. <artifactId>flink-table-runtime</artifactId>
  60. <version>${flink.version}</version>
  61. <scope>provided</scope>
  62. </dependency>
  63. <dependency>
  64. <groupId>org.apache.flink</groupId>
  65. <artifactId>flink-connector-jdbc</artifactId>
  66. <version>3.1.2-1.17</version>
  67. </dependency>
  68. <dependency>
  69. <groupId>mysql</groupId>
  70. <artifactId>mysql-connector-java</artifactId>
  71. <version>8.0.29</version>
  72. </dependency>
  73. </dependencies>
  74. <build>
  75. <plugins>
  76. <plugin>
  77. <groupId>org.apache.maven.plugins</groupId>
  78. <artifactId>maven-shade-plugin</artifactId>
  79. <version>3.2.4</version>
  80. <executions>
  81. <execution>
  82. <phase>package</phase>
  83. <goals>
  84. <goal>shade</goal>
  85. </goals>
  86. <configuration>
  87. <artifactSet>
  88. <excludes>
  89. <exclude>com.google.code.findbugs:jsr305</exclude>
  90. <exclude>org.slf4j:*</exclude>
  91. <exclude>log4j:*</exclude>
  92. <exclude>org.apache.hadoop:*</exclude>
  93. </excludes>
  94. </artifactSet>
  95. <filters>
  96. <filter>
  97. <!-- Do not copy the signatures in the META-INF folder.
  98. Otherwise, this might cause SecurityExceptions when using the JAR. -->
  99. <artifact>*:*</artifact>
  100. <excludes>
  101. <exclude>META-INF/*.SF</exclude>
  102. <exclude>META-INF/*.DSA</exclude>
  103. <exclude>META-INF/*.RSA</exclude>
  104. </excludes>
  105. </filter>
  106. </filters>
  107. <transformers combine.children="append">
  108. <transformer
  109. implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer">
  110. </transformer>
  111. </transformers>
  112. </configuration>
  113. </execution>
  114. </executions>
  115. </plugin>
  116. </plugins>
  117. </build>
  118. </project>
标签: kafka flink

本文转载自: https://blog.csdn.net/weixin_55787608/article/details/141436268
版权归原作者 蚂蚁呀~ 所有, 如有侵权,请联系我们删除。

“flink消费kafka报错”的评论:

还没有评论