This week I mainly worked on setting up monitoring system using prometheus and grafana.

Monitoring and Logging

  • Example of how to write a graylog output plugin
  • Add graylog2 stored data in grafana:
    • Type: elasticsearch
    • Connect to the configured elasticsearch instances used in graylog2
    • Leave 『Index name』empty
    • Time field name: timestamp
  • Prometheus client libraries keep all metrics, do not use keep increasing value as a tag, such as user id.
  • Rsyslog writes logs to file using user and group syslog

Misc