使用 zookeeper 设置 clickhouse 3 节点循环集群时,clickhouse replica/server 无法相互连接
clickhouse replica/server is not able to connect to each other when setting up a clickhouse 3 node circular cluster using zookeeper
数据没有被复制到每个 clickhouse 副本。在检查 clickhouse-server 日志时显示
DB::StorageReplicatedMergeTree::queueTask()::: Poco::Exception。代码:1000,e.code() = 0,e.displayText() = 找不到主机:ip-172-1-140-243(版本 19.9.2.4)
我有 3 台不同的机器,每台机器上都安装了 clickhouse 和 zookeeper。我正在尝试使用 zookeeper 设置 3 节点 clickhouse 集群。我已按照这些步骤进行配置 https://blog.uiza.io/replicated-and-distributed-on-clickhouse-part-2/ 。我已经在所有 clickhouse 实例上创建了表和副本,并在 zookeeper 中进行了验证。在 zookeeper 中为所有副本创建了目录。已在所有实例上创建 /etc/metrica.xml 、 zoo.cfg 、 /etc/clickhouse-server/config.xml 。提供 1 个实例的所有 3 个文件
/etc/metrica.xml 文件
<?xml version="1.0"?>
<yandex>
<clickhouse_remote_servers>
<perftest_3shards_1replicas>
<shard>
<internal_replication>true</internal_replication>
<replica>
<default_database>dwh01</default_database>
<host>172.1.34.199</host>
<port>9000</port>
</replica>
<replica>
<default_database>dwh01</default_database>
<host>172.1.73.156</host>
<port>9000</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<default_database>dwh02</default_database>
<host>172.1.73.156</host>
<port>9000</port>
</replica>
<replica>
<default_database>dwh02</default_database>
<host>172.1.140.243</host>
<port>9000</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<default_database>dwh03</default_database>
<host>172.1.140.243</host>
<port>9000</port>
</replica>
<replica>
<default_database>dwh03</default_database>
<host>172.1.34.199</host>
<port>9000</port>
</replica>
</shard>
</perftest_3shards_1replicas>
</clickhouse_remote_servers>
<zookeeper-servers>
<node index="1">
<host>172.1.34.199</host>
<port>2181</port>
</node>
<node index="2">
<host>172.1.73.156</host>
<port>2181</port>
</node>
<node index="3">
<host>172.1.140.243</host>
<port>2181</port>
</node>
</zookeeper-servers>
<macros replace="replace">
<cluster>OLAPLab</cluster>
<dwhshard00>01</dwhshard00>
<dwhshard01>03</dwhshard01>
<dwhreplica00>01</dwhreplica00>
<dwhreplica01>02</dwhreplica01>
<shard>01</shard>
<replica>node1</replica>
</macros>
<interserver_http_host>ip-172-1-34-199</interserver_http_host>
</yandex>
/etc/clickhouse-server/config.xml
Only added this line rest of the config is default config
<listen_host>::</listen_host>
/usr/lib/zookeeper/conf/zoo.cfg
maxClientCnxns=50
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
# the directory where the transaction logs are stored.
dataLogDir=/var/lib/zookeeper
server.1=172.1.34.199:2888:3888
server.2=172.1.73.156:2888:3888
server.3=172.1.140.243:2888:3888
/etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost6 localhost6.localdomain6
127.0.0.1 ip-172-1-34-199
127.0.0.1 172.1.34.199
我应该更改什么以获取属于所有实例的所有副本中的复制数据
clickhouse 和 zookeeper 端口在机器之间是否可以访问?
比如,你可以 运行 wget http://172.1.140.243:9000
吗?同样适用于 zookeper 端口。
看起来您需要添加到 hosts-两个节点的文件(172.1.34.199 和 172.1 .73.156) 字符串
172.1.140.243 ip-172-1-140-243
数据没有被复制到每个 clickhouse 副本。在检查 clickhouse-server 日志时显示
DB::StorageReplicatedMergeTree::queueTask()::: Poco::Exception。代码:1000,e.code() = 0,e.displayText() = 找不到主机:ip-172-1-140-243(版本 19.9.2.4)
我有 3 台不同的机器,每台机器上都安装了 clickhouse 和 zookeeper。我正在尝试使用 zookeeper 设置 3 节点 clickhouse 集群。我已按照这些步骤进行配置 https://blog.uiza.io/replicated-and-distributed-on-clickhouse-part-2/ 。我已经在所有 clickhouse 实例上创建了表和副本,并在 zookeeper 中进行了验证。在 zookeeper 中为所有副本创建了目录。已在所有实例上创建 /etc/metrica.xml 、 zoo.cfg 、 /etc/clickhouse-server/config.xml 。提供 1 个实例的所有 3 个文件
/etc/metrica.xml 文件
<?xml version="1.0"?>
<yandex>
<clickhouse_remote_servers>
<perftest_3shards_1replicas>
<shard>
<internal_replication>true</internal_replication>
<replica>
<default_database>dwh01</default_database>
<host>172.1.34.199</host>
<port>9000</port>
</replica>
<replica>
<default_database>dwh01</default_database>
<host>172.1.73.156</host>
<port>9000</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<default_database>dwh02</default_database>
<host>172.1.73.156</host>
<port>9000</port>
</replica>
<replica>
<default_database>dwh02</default_database>
<host>172.1.140.243</host>
<port>9000</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<default_database>dwh03</default_database>
<host>172.1.140.243</host>
<port>9000</port>
</replica>
<replica>
<default_database>dwh03</default_database>
<host>172.1.34.199</host>
<port>9000</port>
</replica>
</shard>
</perftest_3shards_1replicas>
</clickhouse_remote_servers>
<zookeeper-servers>
<node index="1">
<host>172.1.34.199</host>
<port>2181</port>
</node>
<node index="2">
<host>172.1.73.156</host>
<port>2181</port>
</node>
<node index="3">
<host>172.1.140.243</host>
<port>2181</port>
</node>
</zookeeper-servers>
<macros replace="replace">
<cluster>OLAPLab</cluster>
<dwhshard00>01</dwhshard00>
<dwhshard01>03</dwhshard01>
<dwhreplica00>01</dwhreplica00>
<dwhreplica01>02</dwhreplica01>
<shard>01</shard>
<replica>node1</replica>
</macros>
<interserver_http_host>ip-172-1-34-199</interserver_http_host>
</yandex>
/etc/clickhouse-server/config.xml
Only added this line rest of the config is default config
<listen_host>::</listen_host>
/usr/lib/zookeeper/conf/zoo.cfg
maxClientCnxns=50
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
# the directory where the transaction logs are stored.
dataLogDir=/var/lib/zookeeper
server.1=172.1.34.199:2888:3888
server.2=172.1.73.156:2888:3888
server.3=172.1.140.243:2888:3888
/etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost6 localhost6.localdomain6
127.0.0.1 ip-172-1-34-199
127.0.0.1 172.1.34.199
我应该更改什么以获取属于所有实例的所有副本中的复制数据
clickhouse 和 zookeeper 端口在机器之间是否可以访问?
比如,你可以 运行 wget http://172.1.140.243:9000
吗?同样适用于 zookeper 端口。
看起来您需要添加到 hosts-两个节点的文件(172.1.34.199 和 172.1 .73.156) 字符串
172.1.140.243 ip-172-1-140-243