One of the returning questions I received recently: why contribute to the syslog-ng upstream? I guess it is a question many open-source projects receive regularly. There are many generic answers. Here I would like to focus more on syslog-ng, focusing on various parts of it.
Of course, the generic answers also apply. Syslog-ng is an open-source project, free to use, modify, and extend. By contributing, you can give something back and improve syslog-ng for everyone. You do not have to be a developer in order to contribute: bug reports, configuration examples, PatternDB rules are all very useful contributions.
C code
Probably the toughest of the contributions. Nowadays C programmers are getting rare. But if you are one of them, and you coded anything (a two line bugfix or a complete new module) for syslog-ng, you should contribute to its upstream. It’s not just that your name is in the release notes and people use your code around the world. There are a lot more technical reasons as well:
-
Your code receives code review. Developers help to make sure that your code does not only comply, but it also conforms to syslog-ng coding style and is efficient.
-
Your code is part of syslog-ng. If a future release has some breaking changes, developers make sure that your code is adapted to these changes.
-
Being part of the syslog-ng code base, your code can be tested together with the rest of syslog-ng. Any code change breaks your feature, so any error is detected immediately, instead of the next time you test.
-
Unless your code has some very special requirements (dependency unavailable in an OS), it comes enabled in packages, ready to use. Creating and distributing your own packages is not an easy task.
-
Other developers can build on your code. Someone contributed an http() destination to syslog-ng. It did its job well, but it was single threaded and without encryption support. Later HTTPS and threading support were added to it, making it one of the most used syslog-ng destinations. Just think of elasticsearch-http(), which is also built around it.
Python code
Most of what I wrote about C also applies to contributing to Python code. The only major difference is that you do not need to recompile syslog-ng to distribute your Python code. But with all the other advantages, it is still worth contributing your code to the syslog-ng upstream. Your code is reviewed, enhanced, maintained, and easily installed together with the rest of syslog-ng.
Configuration snippets
When it comes to syslog-ng, not everything is implemented in C or Python. Many of the features are defined in the syslog-ng configuration. Just think about parsers. JSON parsing, key=value parsing, CSV parsing, etc. are all implemented in C. However, you can combine these parsers into more complex parsers, and parse sudo logs, Apache access logs, Cisco logs, and many more. These are all defined in the syslog-ng configuration library (SCL) and built by combining other parsers.
Another example is the http() destination of syslog-ng, which I have already mentioned. Using the http() destination directly is a kind of DIY. Once you take your time reading the documentation of an HTTP-based API and build a new destination, you should turn it into a configuration block.
In either case, creating a parser or destination is just a first step. Even that is something worth sharing with the community, so nobody needs to reinvent the wheel. But sharing also means that you get help turning it into a proper configuration block. Others can extend it with new possibilities. And if there is a change in syslog-ng, it can be followed up with the rest of the configuration snippets.
PatternDB
This is the odd one out. PatternDB is a technology to parse unstructured log messages, like most syslog messages. Just think of an ssh login message. It requires an XML database to describe log messages and the name-value pairs in them. We have some PatternDB XML examples on GitHub. They are pretty old. I would love to see the collection growing, but it is not easy. There are many problems around it:
-
There is no real standard for naming name-value pairs. There are many competing standards. I even participated in creating a new, common standard, but that initiative died away. So, instead of real standards, all we have are some software specific naming structures.
-
Software is constantly changing. Maintaining a PatternDB XML is a lot more effort than updating code or configuration to work well with the latest syslog-ng version. It needs in-depth knowledge of the software and lots of work.
-
Each user has slightly different expectations. Some parse login events only for selected software, others only the related PAM logs.
-
Parsing and correlation are in the same XML files. Even the parsing can have site-specific differences, but the correlation part is usually completely site-specific. Submitting a version without correlation and maintaining a version with correlation information inside can be lots of extra work.
Still, I think that having more example patterns on GitHub would help others. Even if the proper pattern is not available in the collection for your use case, examples could provide inspiration to get started with this lesser-known but very useful feature of syslog-ng.
If you have a high-traffic central syslog-ng server, then the work of Louise Harding, Fabien Wernli & Co. might help you: automatized pattern creation: https://www.syslog-ng.com/community/b/blog/posts/sequence-making-patterndb-creation-for-syslog-ng-easier
Bug reports
Many people do not consider bug reports a contribution. However, if done properly instead of just writing that “syslog-ng does not work”, then these are at least as useful contributions as the rest I mentioned previously.
Why do I say that? Reproducing a problem is almost half of the problem solving. If you have a proper description of how to reproduce a problem, with configuration and log examples and a detailed description of your environment, it can help pinpoint the problem in the code a lot quicker. Especially if you run syslog-ng in an environment not readily available to syslog-ng developers.
A proper bug report will get you a quicker solution in two ways: developers pick bug reports first which give more information about the problem, and the more information usually also means quicker way to have a proposed fix.
What is next?
The development of syslog-ng follows the rolling release model. Bug fixes are not backported to earlier versions. However, new releases with new features and bug fixes usually follow each other quite quickly. If you would like to contribute a new code or configuration, make sure it works with the latest version of syslog-ng. If you coded for an earlier syslog-ng version, developers can help you update your code or configuration to the latest version.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.