Elasticsearch troubleshooting

Check ES state

curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

Get list of all indices

curl 'localhost:9200/_cat/indices?v'

Check which is in red state
Delete indice

 curl -XDELETE 'http://localhost:9200/graylog2_1251'

When more than 90% space is used -ES starts to move indices and drops indices (sometimes) and crashes into RED state
In logs :
[WARN ][cluster.routing.allocation.decider] [Johann Schmidt] high disk watermark [10%] exceeded on
To disable this "watermark":

 curl -XPUT localhost:9200/_cluster/settings -d '{
    "transient" : {
        "cluster.routing.allocation.disk.threshold_enabled" : false
    }
}'        

              
response


 {"acknowledged":true,"persistent":{},"transient":{"cluster":{"routing":{"allocation":{"disk":{"threshold_enabled":"false"}}}}}}