最近在测试clickhouse集群之前同步数据，暂时没什么好的办法。一位大佬建议使用rsync的方式直接同步文件数据，打算测试下。所以需要快速搭建两个clickhouse集群，测试服务器资源又很有限，所以考虑使用docker compose的方式在单台服务器上快速启动一个clickhouse集群。本文参考https://github.com/tetafro/clickhouse-cluster
镜像，配置文件准备1.涉及的镜像有zookeeper和clickhouse-server，提前拉取好
docker pull zookeeper:3.7docker pull clickhouse/clickhouse-server:22.10.1.18772.主配置文件config.xml （示例是2分片2副本，可以根据实际情况调整）
<?xml version="1.0"?><company> <logger> <level>debug</level> <console>true</console> <log remove="remove"/> <errorlog remove="remove"/> </logger> <query_log> <database>system</database> <table>query_log</table> </query_log> <listen_host>0.0.0.0</listen_host> <http_port>8123</http_port> <tcp_port>9000</tcp_port> <interserver_http_host>clickhouse${REPLICA}</interserver_http_host> <interserver_http_port>9009</interserver_http_port> <max_connections>4096</max_connections> <keep_alive_timeout>3</keep_alive_timeout> <max_concurrent_queries>100</max_concurrent_queries> <uncompressed_cache_size>8589934592</uncompressed_cache_size> <mark_cache_size>5368709120</mark_cache_size> <path>/var/lib/clickhouse/</path> <tmp_path>/var/lib/clickhouse/tmp/</tmp_path> <user_files_path>/var/lib/clickhouse/user_files/</user_files_path> <users_config>users.xml</users_config> <default_profile>default</default_profile> <default_database>default</default_database> <timezone>Europe/Moscow</timezone> <mlock_executable>false</mlock_executable> <remote_servers> <company_cluster> <shard> <replica> <host>clickhouse01</host> <port>9000</port> </replica> <replica> <host>clickhouse02</host> <port>9000</port> </replica> </shard> <shard> <replica> <host>clickhouse03</host> <port>9000</port> </replica> <replica> <host>clickhouse04</host> <port>9000</port> </replica> </shard> </company_cluster> </remote_servers> <zookeeper> <node index="1"> <host>zookeeper</host> <port>2181</port> </node> </zookeeper> <macros> <cluster>company_cluster</cluster> <shard>${SHARD}</shard> <replica>clickhouse${REPLICA}</replica> </macros> <distributed_ddl> <path>/clickhouse/task_queue/ddl</path> </distributed_ddl> <format_schema_path>/var/lib/clickhouse/format_schemas/</format_schema_path></company>3.用户配置文件user.xml，此示例default是空密码，admin密码123
<?xml version="1.0"?><company> <profiles> <default> <max_memory_usage>10000000000</max_memory_usage> <use_uncompressed_cache>0</use_uncompressed_cache> <load_balancing>in_order</load_balancing> <log_queries>1</log_queries> </default> </profiles> <users> <default> <password></password> <profile>default</profile> <networks> <ip>::/0</ip> </networks> <quota>default</quota> </default> <admin> <password>123</password> <profile>default</profile> <networks> <ip>::/0</ip> </networks> <quota>default</quota> </admin> </users> <quotas> <default> <interval> <duration>3600</duration> <queries>0</queries> <errors>0</errors> <result_rows>0</result_rows> <read_rows>0</read_rows> <execution_time>0</execution_time> </interval> </default> </quotas></company>运行docker-compose up前初始化脚本 rm -rf clickhouse01 clickhouse02 clickhouse03 clickhouse04 mkdir -p clickhouse01 clickhouse02 clickhouse03 clickhouse04 REPLICA=01 SHARD=01 envsubst < config.xml > clickhouse01/config.xml REPLICA=02 SHARD=01 envsubst < config.xml > clickhouse02/config.xml REPLICA=03 SHARD=02 envsubst < config.xml > clickhouse03/config.xml REPLICA=04 SHARD=02 envsubst < config.xml > clickhouse04/config.xml cp users.xml clickhouse01/users.xml cp users.xml clickhouse02/users.xml cp users.xml clickhouse03/users.xml cp users.xml clickhouse04/users.xmldocker-compose文件version: '3.5'services: zookeeper: image: zookeeper:3.7 container_name: zookeeper hostname: zookeeper networks: clickhouse-network: ipv4_address: 172.23.0.10 clickhouse01: image: clickhouse/clickhouse-server:22.5 container_name: clickhouse01 hostname: clickhouse01 networks: clickhouse-network: ipv4_address: 172.23.0.11 ports: - "8123:8123" - "127.0.0.1:9000:9000" volumes: - ${PWD}/clickhouse01:/etc/clickhouse-server - ${PWD}/clickhouse01/data01:/var/lib/clickhouse depends_on: - zookeeper clickhouse02: image: clickhouse/clickhouse-server:22.5 container_name: clickhouse02 hostname: clickhouse02 networks: clickhouse-network: ipv4_address: 172.23.0.12 volumes: - ${PWD}/clickhouse02:/etc/clickhouse-server - ${PWD}/clickhouse02/data02:/var/lib/clickhouse depends_on: - zookeeper clickhouse03: image: clickhouse/clickhouse-server:22.5 container_name: clickhouse03 hostname: clickhouse03 networks: clickhouse-network: ipv4_address: 172.23.0.13 volumes: - ${PWD}/clickhouse03:/etc/clickhouse-server - ${PWD}/clickhouse03/data03:/var/lib/clickhouse depends_on: - zookeeper clickhouse04: image: clickhouse/clickhouse-server:22.5 container_name: clickhouse04 hostname: clickhouse04 networks: clickhouse-network: ipv4_address: 172.23.0.14 volumes: - ${PWD}/clickhouse04:/etc/clickhouse-server - ${PWD}/clickhouse04/data04:/var/lib/clickhouse depends_on: - zookeepernetworks: clickhouse-network: name: clickhouse-network ipam: config: - subnet: 172.23.0.0/24数据会持久化到docker-compose所在目录下的clickhouse01-04下的data01-04目录下
PS：在运行的时候可能会出现docker的subenet冲突，修改下subnet网段或者清理对应的subnet
docker network ls和docker network inspect id 查看subnet网段，docker network rm id 删除即可。
启动docker-compose up -d
