Upgrading from syslog-ng open source to premium edition

5 Oct 2017

Upgrading from syslog-ng open source to premium edition

The syslog-ng application has two different editions. Most of my readers use syslog-ng open source edition (OSE). There is also a commercial version of syslog-ng, called the syslog-ng premium edition (PE). It comes with a few extra features, as well as commercial support. Upgrading from OSE to PE or the other way around is not automated and not even always possible. This is due to feature set differences, OSE and PE share a common core but have a different focus. Learn about some of the limitations and some tips about upgrading.

Background

Development of syslog-ng was started by Balázs Scheidler – one of Balabit’s founder – years before Balabit was founded. At that time and for many years that followed, syslog-ng was fully open source. It quickly became part of most Linux distributions and BSD variants. After a while, however, requests for commercial support started coming in, and that’s how syslog-ng PE was born. While both versions are developed mostly by the same set of people, there are also some important differences.

Both syslog-ng OSE and PE users call their software “syslog-ng” without any additional marking. To make our life more simple, I refer to them as “OSE” and “PE”, or simply “syslog-ng” if a statement is valid for both.

OSE – as its name implies – is developed in the open and includes many community-contributed features. Some of these are highly experimental, require exotic external dependencies, or are important only to a very limited set of users. PE is built from the same code base, but includes only a subset of OSE features which are well tested and represent commercial value. These features can be commercially supported as they are covered by automated end-to-end tests which make sure that they not only compile but work correctly on many different platforms. PE also has some exclusive features mostly related to compliance. Contact us if you (plan to) use PE but miss a feature from it which is currently only available in OSE.

Packaging of syslog-ng OSE and PE also varies greatly. With PE, it is easy: all dependencies are included in a single package either in a distribution specific format (rpm or deb) or in a generic .run installer. With OSE, it is completely different. Distribution packages do not bundle dependencies and only include features for which dependencies are available within the distribution. Packaging is modular to make sure that you install only a minimal set of extra dependencies. For example, SQL drivers are only installed if you install the syslog-ng-sql sub package.

To add insult to injury, the naming and content of sub packages varies between distributions, and there are also unofficial OSE packages enabling more features than available in official distribution packages.

What it means for you

Even if you use only basic features in OSE, you will need to edit your syslog-ng.conf to use the correct version number at the top of the file. But most likely you will need to make some more modifications.

As mentioned above, not all OSE features are available in PE. If you try to start PE with an unknown feature enabled, it fails. Packaging can also trigger conflicts, for example systemd service files:

[root@localhost ~]# rpm -Uvh syslog-ng-premium-edition-compact-7.0.5-1.rhel7.x86_64.rpm 
Preparing...                          ################################# [100%]
	file /usr/lib/systemd/system/syslog-ng.service from install of syslog-ng-premium-edition-compact-7.0.5-1.rhel7.x86_64 conflicts with file from package syslog-ng-3.12.1-2.el7.centos.x86_64
[root@localhost ~]#

Upgrading – the clean way

The cleanest way to upgrade from syslog-ng OSE to PE is to remove the OSE package from the system. Unless you did not touch syslog-ng configuration at all, you should of course make a backup of syslog-ng.conf first. This way you can avoid the packaging conflicts and feature differences and do a clean installation of PE.

In my examples below, I upgrade syslog-ng OSE version 3.12 from my unofficial repositories running on Red Hat Enterprise Linux 7.4 to syslog-ng PE version 7.0.4.

Removing OSE

The following instructions assume that the user is in the /root directory.

Copy the contents of /etc/syslog-ng to a directory under /root (or where you can find it…), so you have a backup you can work from later: cp -R /etc/syslog-ng sngose
Remove the syslog-ng package and dependent sub packages: yum erase syslog-ng
Remove the /etc/syslog-ng directory: rm -fr /etc/syslog-ng

Note that you should check the output of yum carefully. If there are any applications listed other than syslog-ng and sub packages, you should rather remove syslog-ng using rpm -e —nodeps, so dependent packages are not removed.

Installing PE

The following instructions assume that the PE rpm package is available in the current directory. You can install syslog-ng PE using the following command:

[root@localhost ~]# rpm -Uvh syslog-ng-premium-edition-compact-7.0.5-1.rhel7.x86_64.rpm 
Preparing...                          ################################# [100%]
Trying to stop syslog services on Linux, using systemd services.
Updating / installing...
   1:syslog-ng-premium-edition-compact################################# [100%]
Created symlink from /etc/systemd/system/multi-user.target.wants/syslog-ng.service to /usr/lib/systemd/system/syslog-ng.service.
[root@localhost ~]#

Merging configurations

The configuration file of the freshly installed PE is available under /opt/syslog-ng/etc/syslog-ng.conf. Before doing anything else, I’d recommend making a backup of it. The next steps are not carved into stone and largely depend on your previous OSE configuration and what you want to achieve:

Append your old OSE configuration to /opt/syslog-ng/etc/syslog-ng.conf
Edit out redundant configuration parts – for example, a version declaration – and those referring to features unavailable in PE – like the Riemann destination.
Syntax check your configuration using the -s option of syslog-ng. Make sure that you use the full path to PE, or add it to the PATH: /opt/syslog-nb/sbin/syslog-ng -s
If no errors are found, stop syslog-ng: systemctl stop syslog-ng
Try to start syslog-ng from the command line in the foreground using the -F option, so you can see any errors:

[root@localhost etc]# /opt/syslog-ng/sbin/syslog-ng -F
[2017-10-03T14:04:18.968550] Error resolving reference; content='source', name='s_sys', location='/opt/syslog-ng/etc/syslog-ng.conf:86:2'

In this case, I forgot to rename a reference to the local system sources. The OSE package used the ‘s_sys’ name for it, the default PE configuration uses ‘s_local’. Once I fixed it, I ran into another problem. As I’m an OSE user, I completely forgot that some features of PE require a license file to be present:

[root@localhost etc]# /opt/syslog-ng/sbin/syslog-ng -F
[2017-10-03T14:07:05.894534] syslog-ng running in client/relay mode, cannot initialize plugin; plugin name='java'
[2017-10-03T14:07:05.894560] Error initializing message pipeline; plugin name='java', location='#buffer:2:3'

All is well that ends well. If your configuration works fine you, do not have to start syslog-ng in the foreground anymore. Stop it using Ctrl-C, and start syslog-ng as a service:
```
systemctl start syslog-ng
```

Future

There are plans at Balabit to make migration easier. Of course not all of the above problems can be eliminated, but still there is room for improvement. Official OSE packages from Balabit will be available soon. As part of the effort, we will try to make sure that these OSE packages are easier to upgrade to syslog-ng PE.