Version 4.6.0 of syslog-ng features not just a new, native log collector for MacOS, but also darwin-oslog-stream(), which can also collect non-persistent log events. Beware that it can collect many megabytes of logs even in just a few minutes! However, if you need to debug MacOS, these logs might come handy.
Before you begin
The good news is that Homebrew already has syslog-ng version 4.6.0 just a few days after release. To install syslog-ng with Homebrew run the following command in your terminal:
brew install syslog-ng
Configuration
There is no public API to collect non-persistent messages, so the darwin-oslog-stream() destination is practically a wrapper around the “log stream” command. You can use the params() option to configure which logs you want to collect. You can read more about the parameters of the log command in the manual page, or run the following command in your terminal to list possible options:
log --help stream
The default value of params() is:
--type log --type trace --level info --level debug
Note that you cannot override the style. By default, it is set to ndjson, as syslog-ng can parse this format and create name-value pairs out of it automatically. You can parse the results further, or simply use the results in your configuration.
You can check what the default configuration will collect using the following command:
log stream --style ndjson --type log --type trace --level info --level debug
It will produce similar log messages on your terminal until you press Ctrl + C:
{"traceID":1375795672689410564,"eventMessage":"Incrementing suppression state to 1 for daemon<com.apple.Finder(503)>","eventType":"logEvent","source":null,"formatString":"Incrementing suppression state to %lu for %{public}@","activityIdentifier":689418,"subsystem":"com.apple.runningboard","category":"monitor","threadID":779063,"senderImageUUID":"A37A3111-A9E4-3AAB-99DB-00BB8BA7A950","backtrace":{"frames":[{"imageOffset":270493,"imageUUID":"A37A3111-A9E4-3AAB-99DB-00BB8BA7A950"}]},"bootUUID":"","processImagePath":"\/usr\/libexec\/runningboardd","timestamp":"2024-01-31 16:35:39.861934+0100","senderImagePath":"\/System\/Library\/PrivateFrameworks\/RunningBoard.framework\/Versions\/A\/RunningBoard","machTimestamp":1570049157720327,"messageType":"Debug","processImageUUID":"E8CD10DE-30CA-3F3E-B255-7363CAF9794B","processID":175,"senderProgramCounter":270493,"parentActivityIdentifier":689413,"timezoneName":""}
For a simple test I just removed a few options from the default, and passed the new settings using params(). The system() source, which uses the native MacOS source, is commented out. Here is the full configuration:
#############################################################################
# Default syslog-ng.conf file which collects all local logs into a
# single file called /var/log/messages.
#
@version: 4.6
@include "scl.conf"
source s_local {
# system();
darwin-oslog-stream(params("--type log --level debug"));
internal();
};
source s_network {
default-network-drivers(
# NOTE: TLS support
#
# the default-network-drivers() source driver opens the TLS
# enabled ports as well, however without an actual key/cert
# pair they will not operate and syslog-ng would display a
# warning at startup.
#
#tls(key-file("/path/to/ssl-private-key") cert-file("/path/to/ssl-cert"))
);
};
destination d_local {
file("/var/log/messages");
file("/var/log/messages-kv.log" template("$ISODATE $HOST $(format-welf --scope all-nv-pairs)\n") frac-digits(3));
};
log {
source(s_local);
# uncomment this line to open port 514 to receive messages
#source(s_network);
destination(d_local);
};
Testing
To test if syslog-ng works correctly, start syslog-ng in the foreground as root with the following command:
sudo /usr/local/sbin/syslog-ng -Fvde
If all goes well, you should see tons of debug messages rolling on your terminal. You can stop syslog-ng by pressing Ctrl + C, and investigate the two log files. /var/log/messages is formatted like a regular syslog log file, while /var/log/messages-kv.log has all the name-value pairs parsed from the original log message. The latter is a lot more verbose. Of course, you do not need all those fields normally, but it is useful to see all the name-value pairs while you are writing your syslog-ng configuration.
Jan 31 16:42:25 balabits-mini systemstats[97]: 0 iterated 3 channels with 3 iterations Jan 31 16:42:27 balabits-mini locationd[327]: 0 (com.apple.locationd.Motion) [Motion] Ping timer fired, resetting watchdog
vs.
2024-01-31T16:42:25.432+01:00 balabits-mini .darwinoslog.activityIdentifier=0 .darwinoslog.backtrace.frames=[{"imageOffset":31695,"imageUUID":"D2C85836-EE99-3E60-974A-A6AA629D1CFD"}] .darwinoslog.bootUUID= .darwinoslog.category= .darwinoslog.eventMessage="iterated 3 channels with 3 iterations" .darwinoslog.eventType=logEvent .darwinoslog.formatString="iterated %d channels with %d iterations" .darwinoslog.machTimestamp=1570454717777368 .darwinoslog.messageType=Debug .darwinoslog.parentActivityIdentifier=0 .darwinoslog.processID=97 .darwinoslog.processImagePath=/usr/sbin/systemstats .darwinoslog.processImageUUID=7271463E-EBD5-340F-BC54-9E65F443DE49 .darwinoslog.senderImagePath=/usr/lib/libIOReport.dylib .darwinoslog.senderImageUUID=D2C85836-EE99-3E60-974A-A6AA629D1CFD .darwinoslog.senderProgramCounter=31695 .darwinoslog.source= .darwinoslog.subsystem= .darwinoslog.threadID=782490 .darwinoslog.timestamp="2024-01-31 16:42:25.432629+0100" .darwinoslog.timezoneName= .darwinoslog.traceID=843153276215230980 .darwinoslog.unixpri=7 HOST=balabits-mini HOST_FROM=balabits-mini MESSAGE="0 iterated 3 channels with 3 iterations" PID=97 PROGRAM=systemstats SOURCE=s_local
2024-01-31T16:42:27.990+01:00 balabits-mini .darwinoslog.activityIdentifier=0 .darwinoslog.backtrace.frames=[{"imageOffset":1299534,"imageUUID":"402F7EAD-17F7-37ED-91DF-2A09BF508BE1"}] .darwinoslog.bootUUID= .darwinoslog.category=Motion .darwinoslog.eventMessage="Ping timer fired, resetting watchdog" .darwinoslog.eventType=logEvent .darwinoslog.formatString="Ping timer fired, resetting watchdog" .darwinoslog.machTimestamp=1570457275955441 .darwinoslog.messageType=Debug .darwinoslog.parentActivityIdentifier=0 .darwinoslog.processID=327 .darwinoslog.processImagePath=/usr/libexec/locationd .darwinoslog.processImageUUID=402F7EAD-17F7-37ED-91DF-2A09BF508BE1 .darwinoslog.senderImagePath=/usr/libexec/locationd .darwinoslog.senderImageUUID=402F7EAD-17F7-37ED-91DF-2A09BF508BE1 .darwinoslog.senderProgramCounter=1299534 .darwinoslog.source= .darwinoslog.subsystem=com.apple.locationd.Motion .darwinoslog.threadID=1817 .darwinoslog.timestamp="2024-01-31 16:42:27.990874+0100" .darwinoslog.timezoneName= .darwinoslog.traceID=21559911225885188 .darwinoslog.unixpri=7 HOST=balabits-mini HOST_FROM=balabits-mini MESSAGE="0 (com.apple.locationd.Motion) [Motion] Ping timer fired, resetting watchdog" PID=327 PROGRAM=locationd SOURCE=s_local
What is next?
Using the default configuration of the darwin-oslog-stream() source uses way too much resources and can fill up your valuable disk space quickly. However, with some experimentation you can narrow down the parameters of the log command to something more manageable. Checking the name-value pairs from the detailed log messages can help you to further fine tune filtering and message routing.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.