syslog-ng documentation

Your main source of knowledge

The syslog-ng product family has an extensive documentation, covering everything from how to install a product to the most complex configuration and settings descriptions. If you cannot find an answer to your question, try the mailing list - our community is always eager to help.

syslog-ng Premium Edition

Contents

7.2. elasticsearch2: Sending messages directly to Elasticsearch version 2.0 or higher The syslog-ng Premium Edition 7 Administrator Guide

Starting with version 5.6 of syslog-ng PE can directly send log messages to Elasticsearch, allowing you to search and analyze your data in real time, and visualize it with Kibana.

Note

In order to use this destination, syslog-ng Premium Edition must run in server mode. Typically, only the central syslog-ng Premium Edition server uses this destination. For details on the server mode, see Section 2.3.3, Server mode.

Note the following limitations when using the syslog-ng PE elasticsearch2 destination:

  • This destination is only supported on the Linux platforms that use the linux glibc2.11 installer, including: Debian 7 (wheezy), Red Hat ES 7, Ubuntu 12.04 (Precise Pangolin), Ubuntu 14.04 (Trusty Tahr).

  • Since syslog-ng PE uses Java libraries, the elasticsearch2 destination has significant memory usage.

  • The log messages of the underlying client libraries are available in the internal() source of syslog-ng PE.

Declaration: 

@module mod-java
@include "scl.conf"

elasticsearch2(
    index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
    type("test")
    cluster("syslog-ng")
);
Example 7.5. Sending log data to Elasticsearch version 2.x and above

The following example defines an elasticsearch2 destination that sends messages in transport mode to an Elasticsearch server running on the localhost, using only the required parameters.

@module mod-java
@include "scl.conf"

destination d_elastic {
  elasticsearch2(
    index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
    type("test")
  );
};

The following example sends 10000 messages in a batch, in transport mode, and includes a custom unique ID for each message.

@module mod-java
@include "scl.conf"

options {
  threaded(yes);
  use-uniqid(yes);
};

source s_syslog {
  syslog();
};

destination d_elastic {
  elasticsearch2(
    index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
    type("test")
    cluster("syslog-ng")
    client-mode("transport")
    custom-id("${UNIQID}")
    flush-limit("10000")
  );
};

log {
  source(s_syslog);
  destination(d_elastic);
  flags(flow-control);
};