0


创建Spark工程

1.新建项目

点击Next

修改名字,点击Finsh

2.添加依赖

  1. <dependencies>
  2. <dependency>
  3. <groupId>org.apache.spark</groupId>
  4. <artifactId>spark-core_2.12</artifactId>
  5. <version>3.0.0</version>
  6. </dependency>
  7. <dependency>
  8. <groupId>org.apache.spark</groupId>
  9. <artifactId>spark-streaming_2.12</artifactId>
  10. <version>3.0.0</version>
  11. </dependency>
  12. <!-- https://mvnrepository.com/artifact/com.alibaba/druid -->
  13. <dependency>
  14. <groupId>org.apache.spark</groupId>
  15. <artifactId>spark-mllib_2.12</artifactId>
  16. <version>3.0.0</version>
  17. </dependency>
  18. </dependencies>

注:Scala版本不一样,依赖也不同,上图为2.12。下图为2.13版本。

  1. <dependencies>
  2. <dependency>
  3. <groupId>org.apache.spark</groupId>
  4. <artifactId>spark-streaming_2.13</artifactId>
  5. <version>3.2.0</version>
  6. </dependency>
  7. <dependency>
  8. <groupId>org.apache.spark</groupId>
  9. <artifactId>spark-core_2.13</artifactId>
  10. <version>3.2.0</version>
  11. </dependency>
  12. <dependency>
  13. <groupId>org.apache.spark</groupId>
  14. <artifactId>spark-mllib_2.12</artifactId>
  15. <version>3.2.0</version>
  16. </dependency>
  17. </dependencies>

下图为Scala2.12添加Spark依赖。

解决加载过慢

  1. <mirrors>
  2. <mirror>
  3. <id>alimaven</id>
  4. <name>aliyun maven</name>
  5. <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
  6. <mirrorOf>central</mirrorOf>
  7. </mirror>
  8. </mirrors>

右下角表示正在加载依赖

3.添加工作框架

** 鼠标右击项目名点击 Add Framewokes Support**

选中Scala,点击OK


本文转载自: https://blog.csdn.net/qq_44035715/article/details/125112538
版权归原作者 卖火柴的蘑菇 所有, 如有侵权,请联系我们删除。

“创建Spark工程”的评论:

还没有评论