This is the fourth part of my syslog-ng tutorial. I hope that since the previous part of my tutorial, you successfully installed syslog-ng. In this part we will finally work with syslog-ng, not just learn about the theoretical background. We will do basic configuration and testing.
You can watch the video or read the text below.
Configuring syslog-ng
Any time I try to explain the syslog-ng configuration, my initial recommendation is: “Don’t Panic” (see: https://en.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27s_Guide_to_the_Galaxy#Don't_Panic). It is simple and logical, even if it looks difficult at first sight (and often also at second sight…). The syslog-ng configuration follows the pipeline model. There are many different building blocks: sources, destinations, filters, parsers, and so on. These building blocks are connected into pipelines using “log” statements in the configuration. You can have multiple pipelines and use the same building blocks in multiple log statements.
/etc/syslog-ng/syslog-ng.conf: getting started
Let me show you a syslog-ng configuration file, which is pretty minimal, but still contains most of the typical configuration elements. It has everything to generate a typical /var/log/messages file from the default log sources of the host.
@version:3.38
@include "scl.conf"
# this is a comment :)
options {flush_lines (0); keep_hostname (yes);};
source s_sys { system(); internal();};
destination d_mesg { file("/var/log/messages"); };
filter f_default { level(info..emerg) and not (facility(mail)); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
We will learn about the various elements in further detail later, in this part I only give you a quick overview. The syslog-ng configuration starts with a version number declaration. It ensures that syslog-ng works as expected even after an upgrade.
You can include other configuration files. SCL in the scl.conf file name stands for Syslog-ng Configuration Library. Many syslog-ng drivers are defined in SCL, not in the code. You need this for example for the Elasticsearch destination.
You can place comments in the configuration with a hashtag.
You can also set global options at the beginning of the configuration. Many of these can be overridden in the configuration later.
Next, you can find a couple of building blocks: a source, a destination and a filter.
The configuration ends with a log statement, which connects the three building blocks together.
SCL: syslog-ng configuration library
As mentioned before in the syslog-ng configuration overview, SCL stands for syslog-ng configuration library. It is a collection of syslog-ng configuration snippets shipped together with syslog-ng. Some drivers are defined here, which work like any other syslog-ng driver from the user’s point of view. Application Adapters, which is the syslog-ng term for automatic message parsing, are also defined here. Some of the more interesting features defined in SCL are the Elasticsearch destination, sudo log parsing, and credit-card number anonymization.
Starting syslog-ng
As syslog-ng normally runs as a service, it starts in the background by default. On a systemd Linux system you can use “systemctl start” to start syslog-ng, and “systemctl stop” to stop it. Over 80% of syslog-ng users use a systemd Linux system, so that is what we use here as an example. Similar tools exist on other Linux and BSD systems.
When working on your syslog-ng configuration, you will use syslog-ng directly. Here are some of the syslog-ng command line options we will use during the syslog-ng tutorial:
-
-s does syntax checking and helps you spot configuration errors before a configuration goes live. Note that it cannot find all problems, like a typo in a source name.
-
-F starts syslog-ng in the foreground
-
-vde provides you with extra information on the terminal about what syslog-ng is doing
-
-f path/to/config allows you to use an alternate configuration instead of the default.
Testing syslog-ng
You will often test new syslog-ng configurations in the foreground. This way you can find configuration problems faster and easier. You can also easily stop syslog-ng using ^C.
There are various tools you can use to test syslog-ng:
-
logger: can send a single message. This utility is usually installed as part of the operating system. The exact command line options vary from system to system.
-
loggen: is a benchmarking and diagnostic tool bundled with syslog-ng. It can generate log messages or read them from files.
Testing steps
Do the testing on a host where logging is not a mission critical function, as logging will often be unavailable while testing.
As a first step, we stop the running syslog implementation. Most likely it is syslog-ng, so,
systemctl stop syslog-ng
should work. Keep in mind, there are some operating systems, which allow multiple syslog implementations to be installed in parallel, so your mileage may vary. Stopping the running syslog allows you to use the same syslog-ng configuration on any operating system to collect local log messages.
The syslog-ng configuration is usually stored in /etc/syslog-ng/syslog-ng.conf or in a similar directory. However, in this tutorial we leave it alone and create other configuration files. Either disable AppArmor or SELinux for syslog-ng, or store your configurations in the same directory as the default syslog-ng configuration.
Using your favorite text editor, save the configuration you have seen at the beginning of this tutorial to a file called first.conf, then start syslog-ng using this configuration in the foreground with debug messages:
syslog-ng -Fvde -f /path/to/first.conf
From another terminal use logger to send a test message:
logger this is a test
When you check the /var/log/messages file, the test message should be there, close to the end of the file:
Jan 6 13:38:16 czplaptop root[16808]: this is a test
If you have any questions or comments, leave a comment on YouTube or reach out to me on Twitter / Mastodon.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.