无法在maven项目中导入graphframes依赖

Question

我有一个 Maven 项目，我需要导入 graphframe 依赖项才能使用 spark grapx，这是我的 pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>demoGraphX</artifactId>
    <version>1.0-SNAPSHOT</version>

    <repositories>
        <repository>
            <id>SparkPackagesRepo</id>
            <url>http://dl.bintray.com/spark-packages/maven</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-graphx_2.11</artifactId>
            <version>2.2.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/graphframes/graphframes -->
        <dependency>
            <groupId>graphframes</groupId>
            <artifactId>graphframes</artifactId>
            <version>0.7.0-spark2.4-s_2.11</version>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.11</artifactId>
            <version>2.2.1</version>
        </dependency>
    </dependencies>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

</project>

但是当我导入 graphframe 依赖项时，我的 IDE return:

Cannot resolve org.apache.parquet:parquet-jackson:1.8.2
Cannot resolve com.twitter:chill_2.11:0.8.0
Cannot resolve org.apache.parquet:parquet-hadoop:1.8.2
Cannot resolve io.dropwizard.metrics:metrics-core:3.1.2
Cannot resolve org.glassfish.hk2:hk2-utils:2.4.0-b34
Cannot resolve com.univocity:univocity-parsers:2.2.1
Cannot resolve org.apache.directory.api:api-asn1-api:1.0.0-M20
Cannot resolve org.glassfish.hk2:osgi-resource-locator:1.0.1
Cannot resolve org.objenesis:objenesis:2.1
Cannot resolve commons-cli:commons-cli:1.2
Cannot resolve org.apache.hadoop:hadoop-yarn-common:2.6.5
Cannot resolve org.mortbay.jetty:jetty-util:6.1.26
Cannot resolve com.google.code.gson:gson:2.2.4
Cannot resolve com.chuusai:shapeless_2.11:2.3.2
etc...

我的 Maven 出现问题。我使用：

Intellij lastest
java 1.8.0_292
Apache Maven 3.6.3

你能帮我解决这个问题吗？谢谢！

Answer 1

bintray 服务从 5 月 1 日起关闭。 (Press release)

因此 Apache spark 社区提供了新的 repo 来托管所有 spark 包。您可以 add/replace 在您的代码中的代码片段下方，应该可以正常工作。

<repositories>
    <repository>
        <id>bintray</id>
        <name>Bintray Repository</name>
        <url>https://repos.spark-packages.org</url>
    </repository>
</repositories>

无法在maven项目中导入graphframes依赖

cannot import graphframes dependency in maven project

java

hadoop

graph

maven

apache-spark