我经过大约370次备份(大约15天)后每小时有一次Elasticsearch备份,我的备份存储库大于15G!但总指数的大小只有500M左右! Elasticsearch是增量备份,但15G VS 500M,差别非常巨大!我想知道索引和备份存储库之间是否有这么大的差异是正常的?备份存储库大小远大于索引大小
是否由我经常备份(每小时)造成的?我使用群集1中的每小时备份和群集2中的每小时还原来实时保持两个ES群集数据相同。
我使用Elasticsearch备份API来备份
设置回购: 卷曲-XPUT “http://IP:9200/_snapshot/backup” -d “{\” 类型\ “:\” FS \”,\ “设置\” :{\ “压缩\”:真,\ “位置\”:\ “备份\”}}”
备份: CURTIME =
date +"%Y-%m-%d %H:%M:%S"
BKTIME = $ {CURTIME // [ - :] /} curl -XPUT“http://IP:9200/snapshot/backup/snapshot $ BKTIME?wait_for_completion = true” 个
我Elasticsearch设置:2个节点,12碎片/节点,2个索引,备份快照存储到NAS
在Elasticsearch数据目录的FS型,指数大小:
node1的索引尺寸: [root @ esnode1 indices] $ du -sh
307M。
节点2指数大小 [根@ esnode2指数] $杜-SH
238M。
[根@ esnode1指数] $杜-LH
8.0K ./index1/10/translog 8.0K ./index1/10/_state 2.9M ./index1/10/index 2.9M ./index1/10 12K ./index1/5/translog 8.0K ./index1/5/_state 1.5M ./index1/5/index 1.5M ./index1/5 8.0K ./index1/4 /超越对 8.0K ./index1/4/_state 2.9M ./index1/4/index 2.9M ./index1/4 8.0K ./index1/_state 8.0K ./index1/7/translog 8.0K ./index1/7/_state 2.9M ./index1/7/index 2.9M ./index1/7 8.0K ./index1/1/translog 8.0K ./index1/1/_state 2.9中号./index1/1/index 2.9M ./index1/1 8.0K ./index1/2/translog 8.0K ./index1/2/_state 2.9M ./index1/2/index 2.9M。 /索引1/2 8.0K ./index1/6/translog 8.0K ./index1/6/_state 3.0M ./index1/6/index 3.0M ./index1/6 8.0K ./index1/0/translog 8.0K ./index1/0/_state 1.5M ./index1/0/index 1.5M ./index1/0 8.0K ./index1/8/translog 8.0K。/索引1/8/_STATE 1.5M ./index1/8/index 1.5M ./index1/8 8.0K ./index1/11/translog 8.0K ./index1/11/_state 2.9M ./index1/11 /索引 2.9M ./index1/11 12K ./index1/9/translog 8.0K ./index1/9/_state 3.0M ./index1/9/index 3.0M ./index1/9 8.0K ./index1/3/translog 8.0K ./index1/3/_state 3.0M ./index1/3/index 3.0M ./index1/3 31M ./index1 16K ./index2/10/ translog 8.0K ./index2/10/_state 16M ./index2/10/index 16M ./index2/10 36K ./index2/5/translog 8.0K ./index2/5/_state 43M ./index2/5/index 43M ./index2/5 20K ./index2/4/超越对 8.0K ./index2/4/_state 17M ./index2/4/index 18M ./index2/4 8.0K ./index2/_state 40K ./index2/7/translog 8.0K ./index2/7/_STATE 32M ./index2/7/index 32M ./index2/7 68K ./index2/1/translog 8.0K ./index2/1/_state 21M ./index2/1/index 21M ./index2/1 64K ./index2/2/translog 8.0K ./index2/2/_state 19M ./index2/2/index 19M ./index2/2 116K ./index2/6/translog 8.0K ./index2/6/_state 22M ./index2/6 /索引 22M ./index2/6 24K ./index2/0/translog 8.0K ./index2/0/_state 17M ./index2/0/index 17M ./index2/0 128K ./索引2/8 /超越对 8.0K ./index2/8/_state 34M ./index2/8/index 34M ./index2/8 72K ./index2/11/translog 8.0K ./index2/11/_state 20M ./index2/11/index 20M ./index2/11 88K ./index2/9/translog 8.0K ./index2/9/_state 22M ./index2/9/index 22M ./index2/9 76K ./index2/3/translog 8.0K ./index2/3/_state 16M ./index2/3/index 16M ./index2/3 277M ./index2 307M。
在备份库,大小: [根@ esnode1备份] $杜-lh
114M ./backup/indices/index1/0
112M ./backup/indices/index1/5
114M ./backup/indices/index1/11
114M ./backup/indices/index1/10
111M ./backup/indices/index1/8
116M ./backup/indices/index1/4
120M ./backup/indices/index1/9
118M。/备份/索引/索引1/3
114M ./backup/indices/index1/2
115M ./backup/indices/index1/7
115M ./backup/indices/index1/1
112M ./backup/indices/index1/6
1.4G ./backup/indices/index1
747M ./backup/indices/index2/0
1.6G ./backup/indices/index2/5
887M ./backup/indices/index2/11
743M ./backup/indices/index2/10
2.1G ./备份/索引/索引2/8
801M ./backup/indices/index2/4
1.3G ./backup/indices/index2/9
8.78 ./backup/ind冰/索引2/3
951M ./backup/indices/index2/2
1.2G ./backup/indices/index2/7
953M ./backup/indices/index2/1
943M ./backup/indices/index2/6
13G ./backup/indices/index2
15G ./backup/indices
15G ./backup
1.1M ./backuplogs
15G。
====== https://www.elastic.co/blog/introducing-snapshot-restore 备份和恢复操作是增量,这意味着只有自上次快照更改的文件将被复制到存储库或恢复成一个索引。 增量快照允许按照需要频繁执行快照操作,而无需太多的磁盘空间开销。用户现在可以轻松地在升级之前创建快照,或者在群集中进行有风险的更改,并在出现问题时快速回滚到以前的索引状态。快照/恢复机制还可用于同步“热”集群和不同地理位置的远程“冷”备份集群之间的数据,以实现快速灾难恢复。
从上面,我的情况是一个真正的问题,任何人都可以帮助我吗?提前致谢 !
你如何触发你的小时备份? – Val
在/ var/spool/cron/root中添加一个linux cron作业 0 1 * * * /opt/es/ESBackup.sh >>/backup/log/backup _ $(date + \%Y - \%m- \ %d).log – caiyufei
您的shell脚本是使用快照/还原还是只是复制数据文件夹?在后一种情况下,请知道,如果不先复制索引,则不应复制数据文件夹,否则会冒着复制不一致数据的风险。 – Val