Running syslog-ng in BastilleBSD

Bastille is a container management system for FreeBSD. If you are coming from a Linux world, it is a bit like Docker or Podman / Buildah from Red Hat, at least some of its functionality. I learned about BastilleBSD right before my Christmas holidays. Currently my primary work platform is Linux and I am just preparing to learn about Kubernetes and Openshift. I planned not to do anything work related during my holidays – which is quite difficult, if your hobby heavily overlaps with your work. Having some strong FreeBSD roots (started to use FreeBSD in 1994), BastilleBSD arrived just on time to be a good excuse to do something IT related :-)

Getting started

Before you begin, make sure that you have FreeBSD 12.1 installed. You should get started by completing the Getting Started guide of BastilleBSD. It helps you to upgrade your system to have the latest security updates, set up the pkg package system and install BastilleBSD. The easiest way is to install it using a package:

pkg install bastille

If you are just testing bastille, it is not really necessary, but you can enable containers automatically at boot with the following command:

sysrc bastille_enable=YES

For added flexibility and security, you can use the ZFS filesystem. As I was testing in a limited virtual environment, I rather skipped this possibility. BastilleBSD works fine without it.

When you come to the packet filter configuration, make sure that you configure the external interface correctly. In my case, I had to change “vtnet0” to “em0”.

Once firewall configuration was ready, I bootstrapped the 12.1 release for my containers:

bastille bootstrap 12.1-RELEASE update

After running this command, your system is finally ready to create containers. I tested all the bastille commands listed in the getting started guide: create, start, list, console, stop and destroy.

Now that you are done with the setup and basic testing, you are ready for the next step: running syslog-ng in bastille!

Syslog-ng in bastille

As I am not too creative when it comes to inventing names and IP addresses, I just used the examples from the getting started guide. If you are more creative, replace “alcatraz” with a better name and “10.17.89.50” with a different IP address.

First, create a new container:

bastille create alcatraz 12.1-RELEASE 10.17.89.50

Here alcatraz is the name of the container, the 12.1-RELEASE is the FreeBSD release it is based on, and at the end of the command line, the IP address is where the container will run. Obviously, if your local network is running on 10.0.0.0/8 IP addresses, then you should rather pick from 172.16.0.0/16 or 192.168.0.0/16.

Start the new container:

bastille start alcatraz

It was a bit of a surprise to me that further configuration only worked when the container was started. Once the container is started, you can start configuring it:

Disable syslogd:

bastille sysrc alcatraz syslogd_enable="NO"

Install syslog-ng:

bastille pkg alcatraz install syslog-ng

Start syslog-ng automagically when starting the container:

bastille sysrc alcatraz syslog_ng_enable="YES"

Edit the syslog-ng configuration of the container. Add a tcp source listening on port 514 and disable any log paths sending logs to /dev/console as it is unavailable in the container. Here is a diff of changes:

root@fb121:~ # diff /usr/local/etc/syslog-ng.conf /usr/local/bastille/jails/alcatraz/root/usr/local/etc/syslog-ng.conf
20c20
< 	     udp(); internal(); };
---
> 	     udp(); tcp(port(514)); internal(); };
92,95c92,95
< log { source(src); filter(f_err); destination(console); };
< log { source(src); filter(f_kern); filter(f_warning); destination(console); };
< log { source(src); filter(f_auth); filter(f_notice); destination(console); };
< log { source(src); filter(f_mail); filter(f_crit); destination(console); };
---
> #log { source(src); filter(f_err); destination(console); };
> #log { source(src); filter(f_kern); filter(f_warning); destination(console); };
> #log { source(src); filter(f_auth); filter(f_notice); destination(console); };
> #log { source(src); filter(f_mail); filter(f_crit); destination(console); };

And finally restart the container for the configuration to take effect:

bastille restart alcatraz

We are almost there. One thing is still missing: firewall configuration. While Linux container tools do this automagically, some extra work is needed when using BastilleBSD. The container is using an IP address on an internal network. If we want to send log messages to port 514 of the container from another host, we need to forward the connection from the public IP address of the host to the internal IP of the container.

Open /etc/pf.conf in your favorite text editor and add a line like this below the commented-out example:

rdr pass inet proto tcp from any to any port {514} -> 10.17.89.50

514 is the port we specified in the syslog-ng configuration and the IP address is what we used when we started the container. You should change any of them if you used a different value earlier. Once you saved it, reload the PF configuration:

service pf restart

Testing

Most likely due to the PF configuration, you cannot test your syslog-ng container from the host running BastilleBSD. You get a “connection refused” when you try to connect to port 514 of the public IP address of your host. So, you need to use a second host for testing. If you do not have syslog(-ng) running there, you can use loggen or telnet to test syslog-ng in BastilleBSD. In my case, the IP address of the host running BastilleBSD is 172.16.167.151. For testing, I used:

telnet 172.16.167.151 514

Just enter some random text, hit enter, and repeat it a couple of times:

Connected to 172.16.167.151.
Escape character is '^]'.
this is a test
another one

Close the connection and check the logs. You should see something similar in /usr/local/bastille/jails/alcatraz/root/var/log/messages:

root@fb121:~ # tail /usr/local/bastille/jails/alcatraz/root/var/log/messages
Jan 10 14:15:38 alcatraz syslog-ng[1732]: Syslog connection accepted; fd='23', client='AF_INET(172.16.167.1:36244)', local='AF_INET(0.0.0.0:514)'
Jan 10 14:15:47 172.16.167.1 this is a test
Jan 10 14:16:12 172.16.167.1 another one

What is next?

Once I was ready with the first version of this blog, I sent it to Christer Edwards – author of BastilleBSD – for a review. Within a few hours, I received some very useful feedback. And not just feedback, he also prepared a template, making the above tasks a lot easier.

The getting started guide includes a section on networking. The method described there involves private IP addresses and port forwarding. It is quite complex, on the other hand it works everywhere, as it does not require additional external IP addresses for the host. There is an easier method as well, which is using IP aliases on the host. Learn more about it from the documentation: https://docs.bastillebsd.org/en/latest/chapters/networking.html

When it comes to testing in my blogs, I always focus on functional testing of syslog-ng. Here are a few more ways to test and troubleshoot the freshly created container:

root@fb121:~ # bastille cmd alcatraz sockstat -4
[alcatraz]:
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     syslog-ng  871   19 udp4   10.17.89.50:514       *:*
root     syslog-ng  871   20 tcp4   10.17.89.50:514       *:*

root@fb121:~ # bastille cmd alcatraz ps -auxw
[alcatraz]:
USER  PID %CPU %MEM   VSZ  RSS TT  STAT STARTED    TIME COMMAND
root  870  0.0  0.3 19256 5340  -  IJ   07:46   0:00.01 /usr/local/sbin/syslog-ng -p /var/run/syslog.pid
root  871  0.0  0.4 23956 8560  -  IsJ  07:46   0:00.16 /usr/local/sbin/syslog-ng -p /var/run/syslog.pid
root  927  0.0  0.1 11408 2560  -  IsJ  07:46   0:00.01 /usr/sbin/cron -J 60 -s
root 1083  0.0  0.1 11684 2720  0  R+J  07:50   0:00.00 ps -auxw

The previous two examples ran troubleshooting tools in the container called alcatraz from the command line of the host. If you expect to run more commands, using the console of the container might be more convenient:

root@fb121:~ # bastille console alcatraz
[alcatraz]:
Last login: Fri Jan 10 14:45:32 on pts/1
FreeBSD 12.1-RELEASE-p1 GENERIC

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
root@alcatraz:~ # sockstat -4
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     syslog-ng  871   19 udp4   10.17.89.50:514       *:*
root     syslog-ng  871   20 tcp4   10.17.89.50:514       *:*

Using BastilleBSD templates

As mentioned earlier, based on the original version of my blog, Christer Edwards prepared a syslog-ng template for BastilleBSD: https://gitlab.com/bastillebsd-templates/syslog-ng This means that instead of building a syslog-ng container from the ground up, you can easily use a template.

First, create a new container, configure the firewall for it if necessary and start it (as mentioned earlier: you can work only on running containers). Next, download the syslog-ng template:

bastille bootstrap https://gitlab.com/BastilleBSD-Templates/syslog-ng

And apply it to the freshly created and started container (replace TARGET with the name of the container):

bastille template TARGET BastilleBSD-Templates/syslog-ng

You should be able to test syslog-ng the same way as described above.

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik.