Data lifecycle

SELKS uses the powerful open source elasticsearch as a database and search engine.

In order to keep the disk space and data in check you can follow these simple steps using elasticsearch-curator.

Curator already comes installed and set up in SELKS:

root@SELKS:/# dpkg -l |grep curator
ii  elasticsearch-curator                 5.8.1                               amd64        Have indices in
Elasticsearch? This is the tool for you!\n\nLike a museum curator manages the exhibits and collections on
display, \nElasticsearch Curator helps you curate, or manage your indices.
root@SELKS:/#

There is also a cronjob that is already set in /etc/crontab/:

0 4 * * * root /opt/selks/delete-old-logs.sh

That will kick in every day at 4 am and clean up/delete all the indices(data) older than 14 days from elsticsearch:

root@SELKS:/# cat /opt/selks/delete-old-logs.sh
#!/bin/bash

/usr/bin/curator_cli delete_indices --filter_list '
[
  {
    "filtertype": "age",
    "source": "creation_date",
    "direction": "older",
    "unit": "days",
    "unit_count": 14
  },
  {
    "filtertype": "pattern",
    "kind": "prefix",
    "value": "logstash*"
  }
]
'