Flink-Cassandra 连接器抛出异常 (flink-connector-cassandra_2.11-1.10.0)

Question

我正在尝试将 flink 1.7.2 升级到 flink 1.10，但我遇到了 cassandra 连接器问题。每次我开始使用它的工作时，都会抛出以下异常：

com.datastax.driver.core.exceptions.TransportException: [/xx.xx.xx.xx] Error writing
    at com.datastax.driver.core.Connection.operationComplete(Connection.java:550)
    at com.datastax.driver.core.Connection.operationComplete(Connection.java:534)
    at com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
    at com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyLateListener(DefaultPromise.java:621)
    at com.datastax.shaded.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:138)
    at com.datastax.shaded.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
    at com.datastax.shaded.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28)
    at com.datastax.driver.core.Connection$Flusher.run(Connection.java:870)
    at com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)
    at com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
    at com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor.run(SingleThreadEventExecutor.java:112)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.datastax.shaded.netty.handler.codec.EncoderException: java.lang.OutOfMemoryError: Direct buffer memory
        at com.datastax.shaded.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:107)
        at com.datastax.shaded.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:643)

当作业运行在本地（不在 YARN 中）时，还会打印以下消息：

13:57:54,490 ERROR com.datastax.shaded.netty.util.ResourceLeakDetector           - LEAK: You are creating too many HashedWheelTimer instances.  HashedWheelTimer is a shared resource that must be reused across the JVM,so that only a few instances are created.

所有不使用 cassandra 连接器的作业都正常工作有人可以帮忙吗？

Answer 1

更新：该错误仍然可以重现，我认为这是原因：https://issues.apache.org/jira/browse/FLINK-17493。

我有一个旧配置（来自 flink 1.7），其中配置了 classloader.parent-first-patterns.additional: com.datastax.，我的 cassadndra-flink 连接器在 flink/lib 文件夹中（这样做是因为与 shaded netty 相关的其他问题我有 Cassandra-flink 连接器）。现在迁移到 flink 1.10 the following problem was hit。删除此配置后 - classloader.parent-first-patterns.additional: com.datastax.，包括 flink-connector-cassandra_2.12-1.10.0.jar 在我的 jar 中并将其从 /usr/lib/flink/lib/ 中删除，问题不再重现。

Flink-Cassandra 连接器抛出异常 (flink-connector-cassandra_2.11-1.10.0)

Flink-Cassandra connector throws exception (flink-connector-cassandra_2.11-1.10.0)

cassandra

apache-flink

flink-streaming