0


创建Spark工程

1.新建项目

点击Next

修改名字,点击Finsh

2.添加依赖

<dependencies>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.12</artifactId>
            <version>3.0.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.12</artifactId>
            <version>3.0.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.alibaba/druid -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_2.12</artifactId>
            <version>3.0.0</version>
        </dependency>

    </dependencies>

注:Scala版本不一样,依赖也不同,上图为2.12。下图为2.13版本。

<dependencies>
<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-streaming_2.13</artifactId>
  <version>3.2.0</version>
</dependency>
<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.13</artifactId>
  <version>3.2.0</version>
</dependency>
<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-mllib_2.12</artifactId>
  <version>3.2.0</version>
</dependency>
</dependencies>

下图为Scala2.12添加Spark依赖。

解决加载过慢

 <mirrors>
        <mirror>
            <id>alimaven</id>
            <name>aliyun maven</name>
            <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
            <mirrorOf>central</mirrorOf>
        </mirror>
    </mirrors>

右下角表示正在加载依赖

3.添加工作框架

** 鼠标右击项目名点击 Add Framewokes Support**

选中Scala,点击OK


本文转载自: https://blog.csdn.net/qq_44035715/article/details/125112538
版权归原作者 卖火柴的蘑菇 所有, 如有侵权,请联系我们删除。

“创建Spark工程”的评论:

还没有评论