Version 3.36 of syslog-ng brings us many interesting new features. There is now basic support for system() source on MacOS, TLS 1.3 ciphers can now be restricted, TLS keylog support was added, symlink creation to the latest file, and there are many new possibilities in syslog parsing.
From this blog, you can learn about some of the new 3.36 features, and we will test symlink creation, which is a community-contributed feature.
Development of syslog-ng 4.0 is already under way, and syslog-ng 3.36 already contains some under-the-hood changes for 4.0. You can learn more about this topic from Balázs Scheidler’s latest blog. Balázs is the original author and still an active developer of syslog-ng: https://syslog-ng-future.blog/syslog-ng-future-the-path-to-syslog-ng-4/
Before you begin
If you want to test any of the features mentioned in this blog, you need at least syslog-ng version 3.36 installed. If it is not yet available as an official update for your operating system, you should check the syslog-ng 3rd-party packages page, available at: https://www.syslog-ng.com/products/open-source-log-management/3rd-party-binaries.aspx
New features
Before going on to testing symlink creation, one of the new features of 3.36 we check in depth, let me share some of the new features of syslog-ng 3.36. For a complete list of new features and bug fixes, you should check the release notes at: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-3.36.1
MacOS support is nothing new. Except for 2-3 years, when ivykis was introduced, syslog-ng compiled just fine on MacOS, at least most parts of it. Since ivykis problems have been resolved, all major functionalities of syslog-ng are available on MacOS again (see https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-macos-monterey for details). What was missing was system() source support. While we still do not have a native driver to fully replace syslogd on a system, the system() source on MacOS can now collect local log messages. It reads the file written by syslogd.
You can now rename name-value pairs by using the new rename() rewrite rule.
The new $(values) and $(names) template functions allow you to query the list of name-value pairs in the current message.
Version-specific cipher suite options were added to the tls() block of syslog-ng. You could limit the cipher suite in earlier syslog-ng versions too, but that only worked for TLS 1.2 and earlier versions. From now on, limiting became version-specific and you can configure it both for TLS 1.2 and earlier and also for TLS 1.3. Here is a configuration example when using tls() with the network destination:
network(
transport("tls")
tls(
pkcs12-file("test.p12")
cipher-suite(
tls12-and-older("ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256"),
tls13("TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384")
)
)
);
Talking about TLS, there is another related feature: syslog-ng can export the TLS symmetric keys for sessions. While most people are reluctant to share the private keys to debug TLS encrypted syslog connections in Wireshark, sharing symmetric keys is less of a privacy concern. It can be shared with others if you need support for debugging a problem.
Log messages often just resemble syslog messages, but do not conform to any standards. There are many workarounds in syslog-ng to make sure that most messages are parsed properly even in this situation. Version 3.36 of syslog-ng added a few features to extend these possibilities even further.
-
transport(text-with-nuls): allows NUL characters in messages when using the network() source of syslog-ng
-
it is now possible to parse ISO dates, where date and time are separated by a space instead of a “T”
-
flags(no-rfc3164-fallback) allows you to drop RFC3164 log messages instead of automatically parsing them when working with RFC5424 log messages
Creating symlinks
One of my favorite new syslog-ng feature comes from the community. Do you use date macros in file names? Then you also spend quite some time finding the latest log file. This is where the new symlink-as() feature of the file() destination can help you. This allows you to have a symlink that always points to the file destination opened most recently.
Note: this feature was planned with date macros in mind. While it works with other macros as well, it is not really practical to enable it for example for application names, as the symlink would be updated too often and too unpredictably.
Configuration
Create a new configuration file with a .conf extension under the /etc/syslog-ng/conf.d/ directory, or append it to syslog-ng.conf:
destination d_mylogs {
file("/var/log/mylogs.${YEAR}${MONTH}${DAY}${HOUR}${MIN}" symlink-as("/var/log/mylogs"));
};
log { source(src); destination(d_mylog); };
In a production environment, log rotation based on the minute does not make much sense (unless you have a really high message rate), but for testing, it is ideal, as you do not have to wait hours or days to see the results. The name of the source is system-specific, the default local log source on a SUSE system is called “src”. Check syslog-ng.conf on your system for the exact name for local logs.
Once you reloaded syslog-ng with the new configuration, you are ready for testing.
Testing
After restarting syslog-ng, you should immediately see a new file and a symlink pointing at it. If you wait for a few minutes, you should see more files and the symlink pointing at the latest file. All of them start with a message about creating a new symlink. If you do not have enough local log messages, you can send some test messages:
logger bla
Here is an example:
czplaptop:/etc/syslog-ng/conf.d # ls -l /var/log/mylogs* lrwxrwxrwx 1 root root 28 Mar 4 13:59 /var/log/mylogs -> /var/log/mylogs.202203041359 -rw-r----- 1 root root 718 Mar 4 13:47 /var/log/mylogs.202203041347 -rw-r----- 1 root root 261 Mar 4 13:48 /var/log/mylogs.202203041348 -rw-r----- 1 root root 2533 Mar 4 13:58 /var/log/mylogs.202203041358 -rw-r----- 1 root root 169 Mar 4 13:59 /var/log/mylogs.202203041359 czplaptop:/etc/syslog-ng/conf.d # cat /var/log/mylogs Mar 4 13:59:34 czplaptop syslog-ng[2126]: Creating symlink; filename='/var/log/mylogs', target='/var/log/mylogs.202203041359' Mar 4 13:59:33 czplaptop root[3510]: bla
What is next?
If you have not done already so: download and install syslog-ng 3.36, at least in a test environment.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik.