流水数据写 ES 压测

压测工具 esrally

  • 编写脚本,生成测试数据 JSONEachRow
esrally --track-path=~/track/10m_kd/ \
 --pipeline=benchmark-only \
 --target-hosts=10.111.3.19:9200,10.111.3.20:9200,10.111.3.21:9200 \
 --offline \
 --report-file=/tmp/report-10m.md

压测报告

报告名词解释

http://esrally.lyremelody.org/zhCN/latest/summaryreport.html

压测方法

  • query-match-all-10k 一次查询 10k 条数据

10,000,000 压测一千万数据(2G)

Metric Task Value Unit
Cumulative indexing time of primary shards 926.319 min
Min cumulative indexing time across primary shards 0 min
Median cumulative indexing time across primary shards 0 min
Max cumulative indexing time across primary shards 189.715 min
Cumulative indexing throttle time of primary shards 0 min
Min cumulative indexing throttle time across primary shards 0 min
Median cumulative indexing throttle time across primary shards 0 min
Max cumulative indexing throttle time across primary shards 0 min
Cumulative merge time of primary shards 444.689 min
Cumulative merge count of primary shards 449
Min cumulative merge time across primary shards 0 min
Median cumulative merge time across primary shards 0 min
Max cumulative merge time across primary shards 342.369 min
Cumulative merge throttle time of primary shards 48.5922 min
Min cumulative merge throttle time across primary shards 0 min
Median cumulative merge throttle time across primary shards 0 min
Max cumulative merge throttle time across primary shards 26.4601 min
Cumulative refresh time of primary shards 162.56 min
Cumulative refresh count of primary shards 7592
Min cumulative refresh time across primary shards 0 min
Median cumulative refresh time across primary shards 0 min
Max cumulative refresh time across primary shards 4.42518 min
Cumulative flush time of primary shards 36.3231 min
Cumulative flush count of primary shards 842
Min cumulative flush time across primary shards 0 min
Median cumulative flush time across primary shards 0 min
Max cumulative flush time across primary shards 12.215 min
Total Young Gen GC 29.81 s
Total Old Gen GC 0 s
Store size 600.827 GB
Translog size 3.4729e-05 GB
Heap used for segments 19.9 MB
Heap used for doc values 8.05996 MB
Heap used for terms 5.51184 MB
Heap used for norms 0.49292 MB
Heap used for points 0 MB
Heap used for stored fields 5.83527 MB
Segment count 6366
Min Throughput bulk 32419.2 docs/s
Median Throughput bulk 34948.8 docs/s
Max Throughput bulk 35597 docs/s
50th percentile latency bulk 544.618 ms
90th percentile latency bulk 852.475 ms
99th percentile latency bulk 6567.99 ms
100th percentile latency bulk 10029.2 ms
50th percentile service time bulk 544.618 ms
90th percentile service time bulk 852.475 ms
99th percentile service time bulk 6567.99 ms
100th percentile service time bulk 10029.2 ms
error rate bulk 0 %
Min Throughput query-match-all 100.02 ops/s
Median Throughput query-match-all 100.03 ops/s
Max Throughput query-match-all 100.06 ops/s
50th percentile latency query-match-all 3.23969 ms
90th percentile latency query-match-all 4.42155 ms
99th percentile latency query-match-all 18.7788 ms
99.9th percentile latency query-match-all 50.5065 ms
100th percentile latency query-match-all 61.5432 ms
50th percentile service time query-match-all 3.13218 ms
90th percentile service time query-match-all 4.19813 ms
99th percentile service time query-match-all 14.7171 ms
99.9th percentile service time query-match-all 43.7854 ms
100th percentile service time query-match-all 61.4653 ms
error rate query-match-all 0 %
Min Throughput query-match-all-10k 4.83 ops/s
Median Throughput query-match-all-10k 4.86 ops/s
Max Throughput query-match-all-10k 4.86 ops/s
50th percentile latency query-match-all-10k 880963 ms
90th percentile latency query-match-all-10k 1.11775e+06 ms
99th percentile latency query-match-all-10k 1.17266e+06 ms
99.9th percentile latency query-match-all-10k 1.18024e+06 ms
100th percentile latency query-match-all-10k 1.18133e+06 ms
50th percentile service time query-match-all-10k 608.404 ms
90th percentile service time query-match-all-10k 749.748 ms
99th percentile service time query-match-all-10k 886.041 ms
99.9th percentile service time query-match-all-10k 1013.36 ms
100th percentile service time query-match-all-10k 1036.93 ms
error rate query-match-all-10k 0 %

20,000,000 压测两千万数据(6G)

Metric Task Value Unit
Cumulative indexing time of primary shards 942.175 min
Min cumulative indexing time across primary shards 0 min
Median cumulative indexing time across primary shards 0 min
Max cumulative indexing time across primary shards 189.715 min
Cumulative indexing throttle time of primary shards 0 min
Min cumulative indexing throttle time across primary shards 0 min
Median cumulative indexing throttle time across primary shards 0 min
Max cumulative indexing throttle time across primary shards 0 min
Cumulative merge time of primary shards 449.05 min
Cumulative merge count of primary shards 453
Min cumulative merge time across primary shards 0 min
Median cumulative merge time across primary shards 0 min
Max cumulative merge time across primary shards 342.369 min
Cumulative merge throttle time of primary shards 49.1307 min
Min cumulative merge throttle time across primary shards 0 min
Median cumulative merge throttle time across primary shards 0 min
Max cumulative merge throttle time across primary shards 26.4601 min
Cumulative refresh time of primary shards 164.416 min
Cumulative refresh count of primary shards 7604
Min cumulative refresh time across primary shards 0 min
Median cumulative refresh time across primary shards 0 min
Max cumulative refresh time across primary shards 4.42518 min
Cumulative flush time of primary shards 36.7088 min
Cumulative flush count of primary shards 848
Min cumulative flush time across primary shards 0 min
Median cumulative flush time across primary shards 0 min
Max cumulative flush time across primary shards 12.215 min
Total Young Gen GC 48.762 s
Total Old Gen GC 0 s
Store size 603.832 GB
Translog size 3.4729e-05 GB
Heap used for segments 20.0072 MB
Heap used for doc values 8.07825 MB
Heap used for terms 5.56839 MB
Heap used for norms 0.499146 MB
Heap used for points 0 MB
Heap used for stored fields 5.86143 MB
Segment count 6400
Min Throughput bulk 34692.8 docs/s
Median Throughput bulk 37088.2 docs/s
Max Throughput bulk 40048.1 docs/s
50th percentile latency bulk 604.131 ms
90th percentile latency bulk 1000.26 ms
99th percentile latency bulk 6933.49 ms
99.9th percentile latency bulk 9321.29 ms
100th percentile latency bulk 10969.2 ms
50th percentile service time bulk 604.131 ms
90th percentile service time bulk 1000.26 ms
99th percentile service time bulk 6933.49 ms
99.9th percentile service time bulk 9321.29 ms
100th percentile service time bulk 10969.2 ms
error rate bulk 0 %
Min Throughput query-match-all 99.96 ops/s
Median Throughput query-match-all 99.99 ops/s
Max Throughput query-match-all 100 ops/s
50th percentile latency query-match-all 18.0637 ms
90th percentile latency query-match-all 22.6536 ms
99th percentile latency query-match-all 25.7546 ms
99.9th percentile latency query-match-all 52.2396 ms
100th percentile latency query-match-all 80.3291 ms
50th percentile service time query-match-all 12.5858 ms
90th percentile service time query-match-all 13.1015 ms
99th percentile service time query-match-all 14.5955 ms
99.9th percentile service time query-match-all 36.1591 ms
100th percentile service time query-match-all 80.0038 ms
error rate query-match-all 0 %
Min Throughput query-match-all-10k 4.8 ops/s
Median Throughput query-match-all-10k 4.83 ops/s
Max Throughput query-match-all-10k 4.84 ops/s
50th percentile latency query-match-all-10k 885599 ms
90th percentile latency query-match-all-10k 1.12163e+06 ms
99th percentile latency query-match-all-10k 1.17445e+06 ms
99.9th percentile latency query-match-all-10k 1.17983e+06 ms
100th percentile latency query-match-all-10k 1.18078e+06 ms
50th percentile service time query-match-all-10k 606.899 ms
90th percentile service time query-match-all-10k 733.711 ms
99th percentile service time query-match-all-10k 903.01 ms
99.9th percentile service time query-match-all-10k 1068.78 ms
100th percentile service time query-match-all-10k 1120.69 ms
error rate query-match-all-10k 0 %

50,000,000 压测两千万数据(14.9gb)

Metric Task Value Unit
Cumulative indexing time of primary shards 1004.26 min
Min cumulative indexing time across primary shards 0 min
Median cumulative indexing time across primary shards 0 min
Max cumulative indexing time across primary shards 189.715 min
Cumulative indexing throttle time of primary shards 0 min
Min cumulative indexing throttle time across primary shards 0 min
Median cumulative indexing throttle time across primary shards 0 min
Max cumulative indexing throttle time across primary shards 0 min
Cumulative merge time of primary shards 481.161 min
Cumulative merge count of primary shards 513
Min cumulative merge time across primary shards 0 min
Median cumulative merge time across primary shards 0 min
Max cumulative merge time across primary shards 342.369 min
Cumulative merge throttle time of primary shards 55.9592 min
Min cumulative merge throttle time across primary shards 0 min
Median cumulative merge throttle time across primary shards 0 min
Max cumulative merge throttle time across primary shards 26.4601 min
Cumulative refresh time of primary shards 170.947 min
Cumulative refresh count of primary shards 7850
Min cumulative refresh time across primary shards 0 min
Median cumulative refresh time across primary shards 0 min
Max cumulative refresh time across primary shards 4.42518 min
Cumulative flush time of primary shards 39.1434 min
Cumulative flush count of primary shards 869
Min cumulative flush time across primary shards 0 min
Median cumulative flush time across primary shards 0 min
Max cumulative flush time across primary shards 12.215 min
Total Young Gen GC 130.536 s
Total Old Gen GC 0 s
Store size 612.732 GB
Translog size 3.4729e-05 GB
Heap used for segments 20.0694 MB
Heap used for doc values 8.07607 MB
Heap used for terms 5.59167 MB
Heap used for norms 0.501709 MB
Heap used for points 0 MB
Heap used for stored fields 5.8999 MB
Segment count 6414
Min Throughput bulk 37718.2 docs/s
Median Throughput bulk 39696.5 docs/s
Max Throughput bulk 47263.3 docs/s
50th percentile latency bulk 1785.51 ms
90th percentile latency bulk 3012.87 ms
99th percentile latency bulk 8610.95 ms
99.9th percentile latency bulk 13259.1 ms
100th percentile latency bulk 14838.4 ms
50th percentile service time bulk 1785.51 ms
90th percentile service time bulk 3012.87 ms
99th percentile service time bulk 8610.95 ms
99.9th percentile service time bulk 13259.1 ms
100th percentile service time bulk 14838.4 ms
error rate bulk 0 %
Min Throughput query-match-all 99.87 ops/s
Median Throughput query-match-all 100.01 ops/s
Max Throughput query-match-all 100.01 ops/s
50th percentile latency query-match-all 3.62666 ms
90th percentile latency query-match-all 4.67795 ms
99th percentile latency query-match-all 90.2625 ms
99.9th percentile latency query-match-all 267.873 ms
100th percentile latency query-match-all 289.898 ms
50th percentile service time query-match-all 3.37023 ms
90th percentile service time query-match-all 4.41682 ms
99th percentile service time query-match-all 17.8509 ms
99.9th percentile service time query-match-all 230.215 ms
100th percentile service time query-match-all 277.346 ms
error rate query-match-all 0 %
Min Throughput query-match-all-10k 4.86 ops/s
Median Throughput query-match-all-10k 4.86 ops/s
Max Throughput query-match-all-10k 4.87 ops/s
50th percentile latency query-match-all-10k 880695 ms
90th percentile latency query-match-all-10k 1.11436e+06 ms
99th percentile latency query-match-all-10k 1.16724e+06 ms
99.9th percentile latency query-match-all-10k 1.17362e+06 ms
100th percentile latency query-match-all-10k 1.17412e+06 ms
50th percentile service time query-match-all-10k 608.759 ms
90th percentile service time query-match-all-10k 734.927 ms
99th percentile service time query-match-all-10k 885.9 ms
99.9th percentile service time query-match-all-10k 1006.49 ms
100th percentile service time query-match-all-10k 1126.4 ms
error rate query-match-all-10k 0 %

配置

track.json

{% import "rally.helpers" as rally with context %}

{
  "version": 2,
  "description": "StackOverflow Q&A stored as nested docs",
  "indices": [
    {
      "name": "sonested",
      "body": "index.json"
    }
  ],
  "corpora": [
    {
      "name": "nested",
      "documents": [
        {
          "source-file": "documents.json",
          "document-count": 50000000,
          "uncompressed-bytes": 10590359571
        }
      ]
    }
  ],
  "challenges": [
    {
      "name": "2-index-and-query",
      "default": true,
      "schedule": [
        {
          "operation": {
            "operation-type": "delete-index"
          }
        },
        {
          "operation": {
            "operation-type": "create-index"
          }
        },
        {
          "operation": {
            "operation-type": "cluster-health",
            "request-params": {
              "wait_for_status": "green"
            }
          }
        },
        {
          "operation": {
            "operation-type": "bulk",
            "bulk-size": 10000
          },
          "warmup-time-period": 120,
          "clients": 8
        },
        {
          "operation": {
            "operation-type": "force-merge"
          }
        },
        {
          "operation": {
            "name": "query-match-all",
            "operation-type": "search",
            "body": {
              "query": {
                "match_all": {}
              }
            }
          },
          "clients": 8,
          "warmup-iterations": 1000,
          "iterations": 1000,
          "target-throughput": 100
        },
    {
          "operation": {
            "name": "query-match-all-10k",
            "operation-type": "search",
            "body": {
          "size": 10000,
              "query": {
                "match_all": {}
              }
            }
          },
          "clients": 3,
          "warmup-iterations": 1000,
          "iterations": 1000,
          "target-throughput": 100
        }
      ]
    }
  ]
}

index.json

{
  "settings": {
    "index.number_of_shards": {{number_of_shards | default(5)}},
    "index.number_of_replicas": {{number_of_replicas | default(0)}},
    "index.store.type": "{{store_type | default('fs')}}",
    "index.requests.cache.enable": false
  },
  "mappings" : {
    "properties" : {
      "cFree" : {
        "type" : "long"
      },
      "cTotal" : {
        "type" : "long"
      },
      "direction" : {
        "type" : "long"
      },
      "free" : {
        "type" : "long"
      },
      "id" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "relId" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "time" : {
        "type" : "long"
      },
      "total" : {
        "type" : "long"
      },
      "type" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "user" : {
        "type" : "long"
      }
    }
  }
}

参考资料

  1. Rally 官方文档

推荐阅读: