This is the eighth part of my syslog-ng tutorial. Last time, we learned about network logging. Today, we learn about syslog-ng macros and templates. At the end of the session, we will know how to do a simple log rotation using macros.
You can watch the video or read the text below.
Macros
Macros are variables defined by syslog-ng. When a syslog message arrives, syslog-ng parses it automatically according to the RFC3164 specification. Macros contain parsed message parts, like date or hostname. There are also many macros that are created by syslog-ng, like the time when a message was received (versus the time parsed from the message), or a macro converted from another macro, like month or day from the date parsed from the message.
Here are some example syslog-ng macros: $FACILITY, $PRIORITY, $DATE, $ISODATE, $YEAR, $MONTH, $WEEK, $DAY, $HOUR, $MINUTE and so on. You can find a lot longer list in the documentation.
In earlier parts of my syslog-ng tutorial, you might have heard me mentioning name-value pairs. How are they different from macros? Name-value pairs are variables defined by any syslog-ng parser or rule, like the CSV parser or a rewrite rule. The difference is minimal, and the two words are often used interchangeably.
Templates
Templates can be used to create new message formats or file names. Templates use macros or name-value pairs combined with static texts. Here is a simple template, which replaces the DATE macro with the more exact ISODATE in messages written to a file. In this case, the template is declared separately, so it can be reused on multiple file destinations.
template t_syslog {
template("$ISODATE $HOST $MSG\n");
};
destination d_syslog {
file("/var/log/syslog" template(t_syslog));
};
Later we will see that templates can be declared inside a file destination. In that case, the template applies only to that single destination.
Templates in file names
In file destinations you can use templates as file names. You can use macros both in the directory and file names. For example: in a central syslog-ng server you can sort incoming log messages based on the host name:
destination t_demo1 {
file("/var/log/$HOST/messages.log" create_dirs(yes));
};
destination t_demo2 {
file("/var/log/$HOST_messages.log");
};
Note the create_dirs(yes) option where the host name is used as a directory name. Without enabling it, the logs are lost if directories are not created for them.
Log rotation
You can do a simple log rotation, which is also based on syslog-ng macros. You can use the various date-related macros in file names. In the example below a new file is started each day for each host:
destination d_messages {
file( “/var/log/$R_YEAR/$R_MONTH/$HOST_$R_DAY.log“ create_dirs(yes));
};
You can create a simple cron job, which compresses and later deletes log files as required by various operational or compliance rules.
If you have any questions or comments, leave a comment on YouTube or reach out to me on Twitter / Mastodon.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.