Google Dataproc 报告的 DataNode 数量不足

Google Dataproc Insufficient number of DataNodes reporting

我使用默认网络配置并尝试 运行 一个有 1 个 master 和 2 个 worker 的标准集群,但它总是失败。工作节点无法执行 RPC 到主节点,反之亦然。我还在集群页面上收到一条信息消息,通知我

The firewall rules for specified network or subnetwork would likely not permit sufficient VM-to-VM communication for Dataproc to function properly

报错信息如下:

Cannot start master: Insufficient number of DataNodes reporting Worker cluster-597c-w-0 unable to register with master cluster-597c-m. This could be because it is offline, or network is misconfigured. Worker cluster-597c-w-1 unable to register with master cluster-597c-m. This could be because it is offline, or network is misconfigured.

虽然我用的是默认配置

所以我发现防火墙规则中有一个配置错误,它允许从 1 到 33535 的 tcp 端口,所以我将其更改为 65535。