阅读比 CPU 核心数量更多的 Kafka 主题
Read more Kafka topics than number of CPU cores
来自Spark Streaming Programming Guide：
Extending the logic to running on a cluster, the number of cores allocated to the Spark Streaming application must be more than the number of receivers. Otherwise the system will receive data, but not be able to process it.
这是否意味着如果我在整个 Spark 集群中有 16 CPU 个核心，我无法从超过 15 个 Kafka 主题中读取数据？
仅当您使用基于 consumer/receiver 的 API 时。这不适用于 Direct Stream。
看看here两者的区别
来自Spark Streaming Programming Guide：
Extending the logic to running on a cluster, the number of cores allocated to the Spark Streaming application must be more than the number of receivers. Otherwise the system will receive data, but not be able to process it.
这是否意味着如果我在整个 Spark 集群中有 16 CPU 个核心，我无法从超过 15 个 Kafka 主题中读取数据？
仅当您使用基于 consumer/receiver 的 API 时。这不适用于 Direct Stream。
看看here两者的区别