Dataflow因java.lang.NoSuchMethodError失败：io.grpc.protobuf.ProtoUtils.marshaller（Lcom/google/protobuf/Message;）

Question

我试图让Dataflow作业在Google Cloud上运行。它总是失败：

java.lang.NoSuchMethodError：io.grpc.protobuf.ProtoUtils.marshaller（LCOM /谷歌/的protobuf /消息;）LIO/GRPC/$的methodDescriptor的Marshaller

这是一个Maven项目，这里是我的依赖：

<dependencies> 
    <dependency> 
    <groupId>com.google.cloud.dataflow</groupId> 
    <artifactId>google-cloud-dataflow-java-sdk-all</artifactId> 
    <version>1.8.0</version> 
    </dependency> 
    <dependency> 
    <groupId>com.google.cloud</groupId> 
    <artifactId>google-cloud-pubsub</artifactId> 
    <version>0.4.0</version> 
    </dependency>  
    <dependency> 
    <groupId>com.google.protobuf</groupId> 
    <artifactId>protobuf-java</artifactId> 
    <version>3.0.0</version> 
    </dependency>  
</dependencies> 

我试过一堆不同的依赖版本。我错过了什么？

无论我通过exec：java还是通过阴影罐运行，结果都是一样的。

完整堆栈跟踪：

(e8dbd0c1b8b8a22): java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lcom/google/protobuf/Message;)Lio/grpc/MethodDescriptor$Marshaller; at com.google.iam.v1.IAMPolicyGrpc.(IAMPolicyGrpc.java:56) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.(PublisherSettings.java:487) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.createDefault(PublisherSettings.java:508) at com.google.cloud.pubsub.spi.v1.PublisherSettings$Builder.access$000(PublisherSettings.java:402) at com.google.cloud.pubsub.spi.v1.PublisherSettings.defaultBuilder(PublisherSettings.java:224) at com.google.cloud.pubsub.spi.DefaultPubSubRpc.(DefaultPubSubRpc.java:138) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubRpcFactory.create(PubSubOptions.java:60) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubRpcFactory.create(PubSubOptions.java:54) at com.google.cloud.ServiceOptions.rpc(ServiceOptions.java:399) at com.google.cloud.pubsub.PubSubImpl.(PubSubImpl.java:115) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubFactory.create(PubSubOptions.java:43) at com.google.cloud.pubsub.PubSubOptions$DefaultPubSubFactory.create(PubSubOptions.java:38) at com.google.cloud.ServiceOptions.service(ServiceOptions.java:391) at com.google.lindsaysmith.titan.DataflowBulkLoadNodes$SendPubSub.sendPubsubMessage(DataflowBulkLoadNodes.java:41) at com.google.lindsaysmith.titan.DataflowBulkLoadNodes$SendPubSub.processElement(DataflowBulkLoadNodes.java:32) at com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:139) at com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:190) at com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47) at com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:55) at

Answer 1

在这种情况下，通常的答案是，你真的必须使用GRPC和协议缓冲的完全版本，在数据流SDK的依赖性声明。这包括所有的传递依赖关系，因此您可能必须禁止其他库的gRPC或protobuf依赖关系，因此它们不会干涉。

您可以看到版本here (gRPC)和here (protobuf)。我将他们从这个答案中删除，以免它过时。