批量插入 Hyperledger Fabric 会超时

Question

我们正在将记录批量插入 Hyperledger Fabric。但是，我们遇到了超时问题。即使我们不断增加超时，我们也只会在稍后发生此错误。

每个事务在所有这些记录的循环中使用 PutState 插入 1000 条记录（盲插入，读取集中没有任何内容）。我们还将 BatchTimeout 增加到 3 秒，将 MaxMessageCount 增加到 100，这样我们就可以得到更大的块（我们看到每个块有 4 个事务，所以每个块都有 4000 [每个事务 4x1000 条记录] 记录被插入到分类帐中）。

当 CouchDB 的 bulk_update 失败并且对等方必须分别重试每个（每个事务的 1000 条记录中的）记录时，查询花费的时间太长并且超时。但这是我们的假设。我们还发现了这个：https://jira.hyperledger.org/browse/FAB-10558，但它说它已经在 v1.2.0 中修复，这是我们正在使用的版本。

我们得到的错误是 net/http: request canceled (Client.Timeout exceeded while reading body) 来自以下日志：

我们尝试在对等容器中设置以下环境变量：

CORE_CHAINCODE_EXECUTETIMEOUT=120s

还有 req.setProposalWaitTime(120 * 1000) 使用 Java SDK 时。

但是，我们稍后会遇到相同的超时错误。所以我们可以继续将超时变量增加到一个更大的数字，但我们相信它会在稍后再次发生。插入 CouchDB 所需的时间是否与 CouchDB 中的记录数成正比？也许当文档数量增加时更新索引会花费更多时间？

我们得到的运行时错误日志（插入2-4百万条记录后）如下：

    October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/core/committer.(*LedgerCommitter).CommitWithPvtData(0xc4222db8c0, 0xc451e4f470, 0xc4312ddd40, 0xdf8475800)
    October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/gossip/state.(*GossipStateProviderImpl).deliverPayloads(0xc4220c5a00)
    October 5th 2018, 04:36:38.646  goroutine 283 [running]:
    October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/core/committer/committer_impl.go:105 +0x6b
    October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/privdata/coordinator.go:236 +0xc3b
    October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/state/state.go:771 +0x6c
    October 5th 2018, 04:36:38.646  
    October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/core/ledger/kvledger.(*kvLedger).CommitWithPvtData(0xc421fb1860, 0xc451e4f470, 0x0, 0x0)
    October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/gossip/privdata.(*coordinator).StoreBlock(0xc422286e60, 0xc42462cd80, 0x0, 0x0, 0x0, 0xc4312dde78, 0x7329db)
    October 5th 2018, 04:36:38.646  github.com/hyperledger/fabric/gossip/state.(*GossipStateProviderImpl).commitBlock(0xc4220c5a00, 0xc42462cd80, 0x0, 0x0, 0x0, 0x0, 0x0)
    October 5th 2018, 04:36:38.646  panic: Error during commit to txmgr:net/http: request canceled (Client.Timeout exceeded while reading body)
    October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/core/ledger/kvledger/kv_ledger.go:273 +0x870
    October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/state/state.go:558 +0x3c5
    October 5th 2018, 04:36:38.646      /opt/gopath/src/github.com/hyperledger/fabric/gossip/state/state.go:239 +0x681
    October 5th 2018, 04:36:38.646  created by github.com/hyperledger/fabric/gossip/state.NewGossipStateProvider
    October 5th 2018, 04:36:03.645  2018-10-04 20:36:00.783 UTC [kvledger] CommitWithPvtData -> INFO 466e[0m Channel [mychannel]: Committed block [1719] with 4 transaction(s)
    October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.807 UTC [statecouchdb] commitUpdates -> WARN 465c[0m CouchDB batch document update encountered an problem. Retrying update for document ID:32216027-da66-4ecd-91a1-a37bdf47f07d
    October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.866 UTC [statecouchdb] commitUpdates -> WARN 4663[0m CouchDB batch document update encountered an problem. Retrying update for document ID:6eaed2ae-e5c4-48b1-b063-20eb3009969b
    October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.870 UTC [statecouchdb] commitUpdates -> WARN 4664[0m CouchDB batch document update encountered an problem. Retrying update for document ID:2ca2fbcc-e78f-4ed0-be70-2c4d7ecbee69
    October 5th 2018, 04:35:56.644  [33m2018-10-04 20:35:55.904 UTC [statecouchdb] commitUpdates -> WARN 4667[0m CouchDB batch document update encountered an problem. ... and so on

Answer 1

[33m2018-10-04 20:35:55.870 UTC [statecouchdb] commitUpdates -> WARN 4664[0m CouchDB batch document update encountered an problem. Retrying update for document ID:2ca2fbcc-e78f-4ed0-be70-2c4d7ecbee69

以上表明 POST http://localhost:5984/db/_bulk_docks 失败，因此分别尝试了各个文档。

查看可供配置的不同参数，在 ledger section 下增加 requestTimeout 可能值得一试。

这可以通过在 docker-compose 中为您的对等容器设置以下环境变量来完成：

CORE_LEDGER_STATE_COUCHDBCONFIG_REQUESTTIMEOUT=100s

与配置参数关联的环境变量的名称可以通过查看答案得出。

配置 CORE_CHAINCODE_EXECUTETIMEOUT 和 proposalWaitime 可能不会产生其他下游连接的影响（这里是 peer 和 couchdb) 超时，然后超时异常向上传播。

批量插入 Hyperledger Fabric 会超时

Bulk insert into Hyperledger Fabric keeps timing out

couchdb

hyperledger

hyperledger-fabric