卡桑德拉。布隆过滤器大小 > 16GB，减少 bloom_filter_fp_chance

Question

将数据恢复到 Cassandra 集群（1 个节点）后出现错误：

ERROR [CompactionExecutor:7] 2016-05-09 08:05:38,621 CassandraDaemon.java:185 - Exception in thread Thread[CompactionExecutor:7,1,main]
java.lang.UnsupportedOperationException: Bloom filter size is > 16GB, reduce the bloom_filter_fp_chance
        at org.apache.cassandra.utils.obs.OffHeapBitSet.<init>(OffHeapBitSet.java:40) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.utils.FilterFactory.createFilter(FilterFactory.java:85) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.utils.FilterFactory.getFilter(FilterFactory.java:78) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.io.sstable.format.big.BigTableWriter$IndexWriter.<init>(BigTableWriter.java:470) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.io.sstable.format.big.BigTableWriter.<init>(BigTableWriter.java:86) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:107) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:84) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.<init>(DefaultCompactionWriter.java:52) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.db.compaction.CompactionTask.getCompactionAwareWriter(CompactionTask.java:237) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:174) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:256) ~[apache-cassandra-2.2.5.jar:2.2.5]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_71]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_71]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_71]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_71]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_71]

任何想法，如何解决？

sstable（数据库文件）的平均大小为 524MB。最大限度。大小为167G

Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 8409389240
Bloom filter off heap memory used: 59948996312

Answer 1

布隆过滤器误报概率与 space 的近似公式为：

m = n * ln(1/fpc)/ln(2)²

m = 字节大小 n = 不同分区键的数量 fpc = 布隆过滤器误报几率

请参阅 http://www.slideshare.net/doanduyhai/cassandra-data-structures-and-algorithms/57 了解数学详细信息

要在大小方面拥有 16Gb 的布隆过滤器，您可能在单个 SSTable（并且可能非常大的 SSTable）中有很多个分区。

请问：

给出你的 SSTables 的平均大小和最大大小？
给导致问题的 table 上配置的布隆过滤器 fp 机会（使用 nodetool cfstats 或 nodetool tablestats）

卡桑德拉。布隆过滤器大小 > 16GB，减少 bloom_filter_fp_chance

Cassandra. Bloom filter size is > 16GB, reduce the bloom_filter_fp_chance

bloom-filter

cassandra