You can read about many interesting syslog-ng features in my blogs. However, it can happen that when you want to try them at home, you fail because the feature is missing. How can you solve such problems? In this blog, I discuss some of the possible solutions from installing sub-packages through using unofficial repositories, to upgrading your OS.
This blog focuses on RPM packages for openSUSE / SLES, Fedora / RHEL, and FreeBSD, because these are the packages I know – I am their maintainer. However, these problems and their solutions also apply to Debian / Ubuntu, and other Linux distributions.
Installing sub-packages
The most common reason for a missing feature is that when you install syslog-ng, you only install its core features. Everything (or almost everything) that requires an extra dependency is only available through sub-packages.
For example when you search for syslog-ng, you will see a list similar to this:
[root@localhost ~]# dnf search syslog-ng | grep -v debuginfo ======================= Name Exactly Matched: syslog-ng ======================== syslog-ng.x86_64 : Next-generation syslog server syslog-ng.src : Next-generation syslog server ====================== Name & Summary Matched: syslog-ng ======================= syslog-ng-afsnmp.x86_64 : SNMP support for syslog-ng syslog-ng-amqp.x86_64 : AMQP support for syslog-ng syslog-ng-debugsource.x86_64 : Debug sources for package syslog-ng syslog-ng-devel.x86_64 : Development files for syslog-ng syslog-ng-geoip.x86_64 : geoip support for syslog-ng syslog-ng-http.x86_64 : HTTP support for syslog-ng syslog-ng-java.x86_64 : Java destination support for syslog-ng syslog-ng-kafka.x86_64 : kafka support for syslog-ng syslog-ng-libdbi.x86_64 : Libdbi support for syslog-ng syslog-ng-logrotate.x86_64 : Logrotate script for syslog-ng syslog-ng-mongodb.x86_64 : mongodb support for syslog-ng syslog-ng-mqtt.x86_64 : mqtt support for syslog-ng syslog-ng-python.x86_64 : Python support for syslog-ng syslog-ng-python-modules.x86_64 : Python-based drivers for syslog-ng syslog-ng-redis.x86_64 : redis support for syslog-ng syslog-ng-riemann.x86_64 : riemann support for syslog-ng syslog-ng-slog.x86_64 : $(slog) support for syslog-ng syslog-ng-smtp.x86_64 : smtp support for syslog-ng syslog-ng-snmpdest.x86_64 : SNMP destination support for syslog-ng syslog-ng-sql.x86_64 : SQL support for syslog-ng
All of these sub-packages contain a syslog-ng module, which installs an extra dependency. In some cases it is just a small package, only a few kilobytes. However, Java and Python support pull in hundreds of megabytes of extra dependencies. The only exception is JSON support, as it is part of the base syslog-ng package in all of my official and unofficial packages.
The typical symptom of a missing feature is that syslog-ng complains that a module is missing. The most frequent problem report I receive is that someone wants to send logs to Elasticsearch, configures the elasticsearch-http() destination, but syslog-ng fails to start:
[root@localhost conf.d]# syslog-ng -s
[2023-09-22T13:54:27.465048] Cannot load required module; module='http', details='The elasticsearch-http() driver depends on the syslog-ng http module, please install the syslog-ng-mod-http (Debian & derivatives) or the syslog-ng-http (RHEL & co) package', location='/usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:27:2'
Error parsing config, syntax error, unexpected LL_ERROR, expecting '}' in /usr/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf:27:1-27:2:
22
23 @requires json-plugin
24
25 block destination elasticsearch-http(
26 url()
27----> index()
27----> ^
28 type("")
29 custom_id("")
30 workers(4)
31 batch_lines(100)
32 timeout(10)
Included from /etc/syslog-ng/conf.d/elastic.conf:2:5-12:6:
1 destination d_elasticsearch_http {
2-----> elasticsearch-http(
2-----> ^^^^^^^^^^^^^^^^^^^^
3-----> index("syslog-ng")
4-----> type("")
5-----> user("elastic")
6-----> password("Gr3Cmh7-5CdZYGl+uCWB")
7-----> url("https://localhost:9200/_bulk")
Included from /etc/syslog-ng/syslog-ng.conf:68:1-68:1:
63 log { source(s_sys); filter(f_cron); destination(d_cron); };
64
65
66 # Source additional configuration files (.conf extension only)
67 @include "/etc/syslog-ng/conf.d/*.conf"
68---->
68----> ^
69
70 # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:
syslog-ng documentation: https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition
contact:
GitHub Project: https://github.com/syslog-ng/syslog-ng
Chat with the Developers: https://gitter.im/syslog-ng/syslog-ng
Mailing List: https://lists.balabit.hu/mailman/listinfo/syslog-ng
In this case, syslog-ng even helps you by telling you to install a missing module. You do not see an explanation in each case, but if a source or destination driver is unknown by syslog-ng, check if the related module is installed on your host.
Using unofficial repositories
Except for rolling Linux distributions, the version of syslog-ng in a Linux distribution stays at the version of syslog-ng available at the time of release. If the Linux distribution you use is nine years old (such as SLES 12 and RHEL 7), this means that the bundled syslog-ng version is nine years old, in which, for example, HTTP support was not yet available. But you do not need to go back nine years for missing features: OpenTelemetry support was added just two months ago.
In other cases, you can have a recent enough syslog-ng version as part of your favorite Linux distribution but still miss a syslog-ng feature if its dependency is unavailable in the distribution. Linux distributions are closed systems: you can only use packages available in the distro to build a package for the distro.
In either case using one of my unofficial RPM repositories can help you. This way you can install up-to-date syslog-ng packages on your distribution from third-party repositories, which also include some of the missing dependencies.
There are several flavors of these repositories, but I do not want to go into details here. You can learn more about them at: https://www.syslog-ng.com/community/b/blog/posts/overview-of-syslog-ng-rpm-repositories
Upgrading your OS
There are also situations when using an unofficial repository on your current OS does not help. For example OpenTelemetry support is not available for RHEL, only for Fedora or openSUSE Tumbleweed. The reason is simple: while adding a missing dependency to a third-party repository does not have any risks, adding a new version of a core system library can cause significant troubles in other parts of the system.
In this case, you have few possibilities. You can install syslog-ng on an OS which supports the missing feature. Note that the OS might be cutting edge, without long term and / or enterprise support. But you can also keep your old OS and run syslog-ng in a container instead. The syslog-ng container on Github has almost all features enabled: https://www.syslog-ng.com/community/b/blog/posts/nightly-syslog-ng-container-images
Compiling on FreeBSD
Packages on FreeBSD are built from FreeBSD ports with the default options of the port. These options try to keep a careful balance: by enabling the most commonly used features of an application without pulling in too many additional dependencies. In case of syslog-ng, this means that JSON and HTTP support are part of the package: you can parse incoming JSON-formatted messages, and then can use the JSON template to send logs to Elasticsearch or to various cloud services using the http() destination.
If you want to use Python support or some of the less common destinations, like Riemann, you have to compile syslog-ng yourself from ports. Almost all syslog-ng features are available on FreeBSD if you compile it from ports – but not everything. Linux-specific drivers are obviously missing, like speeding up UDP log collection using BPF. Drivers written in C++, like OpenTelemetry, are also missing. Those require GCC, and the syslog-ng ./configure script on FreeBSD fails if we switch the compiler to GCC.
What is next?
With a bit of luck, all you have to do is to extend your syslog-ng installation with a not yet installed sub-module. However, if your OS or syslog-ng installation on your system is too old, you might need some more work to be done. In a simple case you can use a third-party repository to install an up-to-date syslog-ng version. But you might also need to switch to a more up-to-date OS or use containers.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.