Dataflow fails with java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lcom/google/protobuf/Message;)

Question

我正在尝试在 Google 云上将数据流作业发送到 运行。它总是失败：

java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lcom/google/protobuf/Message;)Lio/grpc/MethodDescriptor$Marshaller

这是一个 Maven 项目，这是我的依赖项：

<dependencies>
  <dependency>
    <groupId>com.google.cloud.dataflow</groupId>
    <artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
    <version>1.8.0</version>
  </dependency>   
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-pubsub</artifactId>
    <version>0.4.0</version>
  </dependency>    
  <dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>3.0.0</version>
  </dependency>       
</dependencies>

我尝试了很多不同的依赖版本。我错过了什么？

无论是 运行 通过 exec:java 还是通过遮阳罐，结果都是一样的。

完整堆栈跟踪：

(e8dbd0c1b8b8a22): java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lcom/google/protobuf/Message;)Lio/grpc/MethodDescriptor$Marshaller; at com.google.iam.v1.IAMPolicyGrpc.(IAMPolicyGrpc.java:56) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.(PublisherSettings.java:487) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.createDefault(PublisherSettings.java:508) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.access[=11=]0(PublisherSettings.java:402) at com.google.cloud.pubsub.spi.v1.PublisherSettings.defaultBuilder(PublisherSettings.java:224) at com.google.cloud.pubsub.spi.DefaultPubSubRpc.(DefaultPubSubRpc.java:138) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubRpcFactory.create(PubSubOptions.java:60) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubRpcFactory.create(PubSubOptions.java:54) at com.google.cloud.ServiceOptions.rpc(ServiceOptions.java:399) at com.google.cloud.pubsub.PubSubImpl.(PubSubImpl.java:115) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubFactory.create(PubSubOptions.java:43) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubFactory.create(PubSubOptions.java:38) at com.google.cloud.ServiceOptions.service(ServiceOptions.java:391) at com.google.lindsaysmith.titan.DataflowBulkLoadNodes$SendPubSub.sendPubsubMessage(DataflowBulkLoadNodes.java:41) at com.google.lindsaysmith.titan.DataflowBulkLoadNodes$SendPubSub.processElement(DataflowBulkLoadNodes.java:32) at com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:139) at com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:190) at com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47) at com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:55) at

Answer 1

在这种情况下，通常的答案是您确实必须使用在 Dataflow SDK 的依赖项中声明的 gRPC 和 Protocol Buffers 的确切版本。这包括所有传递依赖项，因此您可能必须抑制其他库的 gRPC 或 protobuf 依赖项，以免它们干扰。

您可以看到版本 here (gRPC) and here (protobuf)。我将他们排除在这个答案之外，这样它就不会过时。