This is the seventh part of my syslog-ng tutorial. Last time, we learned about syslog-ng destinations and the log path. Today, we learn about syslog-ng network logging. At the end of the session, we will send test messages to a syslog-ng network source.
You can watch the video or read the text below.
RFC3164
When it comes to syslog, most people still think about RFC3164, which is also often called legacy syslog. It is old, not really well-standardized, as it just tries to describe existing practice. Still, most syslog messages arrive in this format.
A typical RFC3164 syslog message looks like this:
<123>Aug 1 10:28:22 host syslog-ng[12446]: syslog-ng starting up; version='4.0.1'
It has three parts: <PRI>HEADERS: MESSAGE
-
<PRI> is calculated from facility (which is the category of the sender application) and severity (which declares the importance of the message). This information is used sometimes for filtering log messages, but normally does not appear in log messages saved to files.
-
HEADERS include a timestamp, a hostname, and often a program name and related process ID: Aug 1 10:28:22 host syslog-ng[12446]:
-
The rest of the line is the actual message. For example: syslog-ng starting up; version='4.0.1'
RFC5424
RFC5424 is a well-standardized format for syslog messages, right from the beginning. It has a more precise timestamp, and can forward name-value pairs. However, it is not widely used.
What we can see a lot more often is that if someone wants to forward name-value pairs between syslog servers, they use a legacy RFC3164 syslog header, and a JSON formatted message part.
Modes of operation
When it comes to collecting log messages over a network, syslog-ng has three modes of operation:
-
Client mode: syslog-ng is collecting logs from the client and sending them to the central server directly or through a relay.
-
Relay mode: syslog-ng is collecting logs from clients through the network and sending them to the central server directly or through another relay.
-
Server mode: syslog-ng is collecting logs from clients and / or relays and storing them either locally or in a non-syslog destination-driver.
Of course, in the real world, these modes are not so strictly separated. I mean, some relays might both store and forward log messages.
Why relays?
Relays have many important roles in a logging infrastructure. Many devices use UDP for log transport. UDP is an unreliable protocol, so you want to collect these log messages as close to the source as possible. Later, we will learn about message parsing, which can be resource-hungry. Using relays, you can distribute processing, so not everything has to be done at the central server.
Using relays can give structure and additional security to your logging infrastructure. You can install a relay for each department or site. This is especially important when the central server is at a remote location. Relays ensure that log messages leave clients immediately even if the central server is unavailable due to maintenance or network problems.
syslog-ng.conf: netsource.conf
The following syslog-ng configuration implements a very simple syslog-ng server. Save this as netsource.conf on your test machine.
@version:3.38
source s_tcp { tcp(port(514)); };
destination d_file { file("/var/log/fromnet"); };
log { source(s_tcp); destination(d_file); };
Stop the running syslog implementation and start syslog-ng with this configuration in the foreground with debug information enabled:
syslog-ng -Fvde -f /etc/syslog-ng/netsource.conf
Using logger with a network source
From another terminal, you can now use the logger command to generate test messages. Note that the exact parameters of logger might be different on your system:
logger -T -n 127.0.0.1 -P 514 this is a test message
Important options
-
-T: TCP
-
-n: hostname or IP
-
-P: port
-
Log message
With syslog-ng started in the foreground and with debugging enabled, you should see the incoming log message on the screen. The file /var/log/fromnet should show your test message at the end.
If you have any questions or comments, leave a comment on YouTube or reach out to me on Twitter / Mastodon.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.