When a user reports a problem with syslog-ng, developers create code to fix it. The fix is called a patch or a pull request (PR) in Git terminology. Often this code does not enter the main source code automatically but developers ask the user to test the fix before the code is merged. A new challenge starts here: figuring out how to test a proposed fix. My blog shows you how to download and patch the syslog-ng source code and build easy-to-install (and remove) RPM packages from it. I did it on CentOS 7, but you should be able to use my instructions with minimal modifications on RHEL 7 or Fedora as well.

Creating a generic developers’ guide is beyond the scope of this blog. Instead of that I will show you how to check out the latest syslog-ng sources from GitHub, the server where the syslog-ng source code is maintained. It is easier to test fixes or new features if syslog-ng is available as a package. Creating an RPM package from scratch is difficult, so I instead re-use the source RPM of my unofficial syslog-ng package together with the dependencies that are required to build it.

You will learn to take the following steps in the procedures described below:

1. Preparing your environment for downloading syslog-ng sources.

2. Downloading syslog-ng sources and applying the proposed fix for your issue.

3. Creating a source Tarball using the patched syslog-ng source.

4. Downloading and extracting a source RPM, and updating it with the source Tarball

5. Downloading extra dependencies that are required for building the new RPM package.

6. Building the new source RPM package.

7. Building the new RPM packages from the source RPM.

8. Cleaning up your machine.

For information on other platforms that could be relevant to you, check out all my blog posts about installing syslog-ng on major Linux distributions, collected in one place.

In addition, syslog-ng is also available as a Docker image. To learn more, read our tutorial about logging in Docker using syslog-ng.

Preparing your environment

First of all, you have to install a few applications. Some of these are not available in CentOS or RHEL. All of these are available in Fedora Linux.

1. Depending on whether you have RHEL or CentOS 7, do the following:

• On RHEL: Enable the so-called “optional” repository that contains a number of packages that are required to start syslog-ng.

• On CentOS: The content of this repository is included in CentOS, so you do not have to enable it separately.

2. The Extra Packages for Enterprise Linux (EPEL) repository contains many useful packages that are not included in RHEL or CentOS. A few dependencies of syslog-ng are available in this repository. To enable it, download and install the following RPM package:

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm  

rpm -Uvh epel-release-latest-7.noarch.rpm

3. Install packages that are required for downloading syslog-ng sources and creating packages from them:

yum install docker git patch mock

4. Start Docker:

systemctl start docker

To start Docker automatically on next boot, enter the following command:

systemctl enable docker

You now have all software that are required to download syslog-ng sources and create packages from it available on your system.

Note: Because one of the tools (Mock) cannot be started as root, run the rest of the commands as a regular user.

Downloading and patching syslog-ng sources

In the following you will learn how to download the syslog-ng sources from Git, find a patch that you are interested in, download it and then apply it to the syslog-ng source code.

1. Download syslog-ng sources using Git:

git clone https://github.com/balabit/syslog-ng

This will create a new sub-directory – called syslog-ng – in your current working directory and place the source code there.

2. Once the download is finished, change to this new sub-directory:

cd syslog-ng

3. When the developers provide a fix for your issue or create an enhancement based on your request, they will provide you with a URL pointing to this fix or enhancement, called a pull request (PR). The following is an example PR for an exciting new syslog-ng feature: https://github.com/balabit/syslog-ng/pull/1856

As you can see, a PR is not only a patch with a new feature, but more than that. A PR is more like a description and a discussion of the patch. If you want to download the patch, append .patch to the URL of the PR and your browser will be redirected to a patch file. For the previous example, the URL will look like the following: (https://patch-diff.githubusercontent.com/raw/balabit/syslog-ng/pull/1856.patch)

To download the patch from command line, for example with wget, enter the following command:

wget https://patch-diff.githubusercontent.com/raw/balabit/syslog-ng/pull/1856.patch

4. Apply the patch (modify the syslog-ng source code using the patch downloaded) using the patch utility:

patch -p1 < 1856.patch

Creating a source Tarball

Developers usually work on Git sources. These often require installing additional software and packages, and therefore these are more time-consuming or difficult to use. On the other hand, Linux distributions and end-users usually use something called the source Tarball. The source Tarball includes much more than a simple syslog-ng source, to make the life of end-users easier. Along with syslog-ng sources, it also contains sources for some of the dependencies, it has generated man pages (help documentation and user manual), a configure script that you can use to configure which syslog-ng features are enabled, and a few more other things.

You might have wondered why you had to install Docker. There is a tool in the syslog-ng Git sources that utilizes Docker to make sure that source Tarball generation happens in a well-defined environment with all the required tools and dependencies available.

To build the source Tarball, you do not need to have in-depth Docker knowledge, because it only requires issuing a single command. Issue the following command from the syslog-ng source code directory:

sudo dbld/rules tarball

You will find the generated source Tarball in the dbld/build directory. Right now (until the next syslog-ng release that will modify the version number) the name of the generated file is syslog-ng-3.13.2.tar.gz

Downloading and extracting a source RPM

The source RPM is an RPM file that contains the source Tarball, patches, a sample configuration and the .spec file that controls package building. This is used as an input to build RPM packages for installation. I regularly build RPM packages from syslog-ng Git sources. You can use my syslog-ng source RPM file with minimal or no modifications to build your own RPM packages. It is available in my Copr (Fedora package builder service) project at https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/

While I could give you a direct URL to the current source RPM, it can change any time I publish an update. Therefore, I will explain how to find the source RPM instead.

1. Enter my Copr project URL in your browser: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/

2. Select the Packages tab.

3. Select syslog-ng under the Name column.

4. Select the Build ID (for example, 654321) of the most recent build (that was Submitted most recently).

5. Scroll down and select epel-7-x86_64 under the Chroot Name column.

6. Download the file that ends with src.rpm (currently it is syslog-ng-3.13.2-3.el7.centos.src.rpm).

Once you have downloaded the source RPM, complete the following steps:

1. To create a new directory (for example, “sngbuild”), enter:

mkdir sngbuild

2. To change to the new directory, enter:

cd sngbuild

3. To extract the content of the downloaded file, enter:

rpm2cpio ../syslog-ng-3.13.2-3.el7.centos.src.rpm | cpio --extract --make-directories --verbose

Change the package name as required, if the package that you have downloaded has a different name (for example, because of version change).

4. To check the contents of the directory, enter:

ls

5. You will see that there is already a syslog-ng source Tarball. Delete that source Tarball, because you want to build a new RPM using the sources you have just prepared. To delete the source Tarball, enter:

rm syslog-ng-3.13.2.tar.gz

6. Copy the syslog-ng tarball prepared in the “Creating a source Tarball” procedure to the current directory. The source path will be something similar as in the next command. Change the path name and file name as required:

cp /path/to/syslog-ng/dbld/build/syslog-ng-3.13.2.tar.gz .

Downloading extra dependencies

Not all dependencies of the syslog-ng package are included in CentOS or EPEL. To be able to compile syslog-ng with all features enabled, you have to download these from the same location as described in the Downloading and extracting a source RPM procedure.

You will need RPM packages for Gradle, libdbi and libdbi-devel and syslog-ng-java-deps (or in case of Fedora, only syslog-ng-java-deps). Create a new sub-directory with the name “dep” (short for dependencies) and save the files there.

1. Enter my Copr project URL in your browser: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/

2. Select the Packages tab.

3. Select gradle / libdbi / libdbi-drivers / syslog-ng-java-deps under the Name column, respectively.

4. Select the Build ID (for example, 654321) of the most recent build (that was Submitted most recently).

5. Scroll down and select epel-7-x86_64 under the Chroot Name column.

6. Download the file withthat name ends with src.rpm (currently it is <package-name>.centos.src.rpm).

When you have finished downloading these extra dependencies, you will have the following files (or similar) in the dep directory:

gradle-2.6-1.el7.centos.x86_64.rpm  libdbi-0.9.0-6.el7.centos.x86_64.rpm  libdbi-devel-0.9.0-6.el7.centos.x86_64.rpm  syslog-ng-java-deps-1.0-2.noarch.rpm

Building the new source RPM

The first step of building RPM packages is creating a source RPM. In the Downloading and extracting a source RPM procedure we have downloaded one such package and extracted it to be able to replace the included source code. Before we actually build the source RPM there is one more task: checking version numbers. The version number in the source Tarball file name must match the “Version:” field in the syslog-ng.spec file. You can find it near to the top of the file. If there is any difference, correct the value in the .spec file.

Once your .spec file is ready and has the correct version number, you can create the source RPM. To create the source RPM, navigate to the directory containing the syslog-ng.spec file and enter the following command:

mock -v -r epel-7-x86_64 --buildsrpm --resultdir=work --spec syslog-ng.spec --sources .

This command does not look friendly and nice at first sight, so I better explain it  “Mock” is a tool to build RPM packages in a clean, chrooted environment, which only contains a minimal set of development tools and packages defined in the .spec file. This ensures that there are no missing or accidental additional dependencies. It has several command line options, we will use the following ones in our case:

• -v is optional and runs Mock in verbose mode, which makes it easier to detect and catch bugs.

• -r epel-7-x86_64 defines the target distribution, its version and architecture. We target here RHEL 7 on 64bit x86, but thanks to using an isolated environment it is possible to compile for other distributions and versions too. For a complete list check the /etc/mock directory.

• –buildsrpm builds a source RPM package.

• –resultdir=work saves the logs and the resulting source RPM into a sub-directory called “work”. It is optional but helps to keep the source directory clean.

• –spec syslog-ng.spec defines the .spec file to be used for source RPM building.

• –sources . defines where Mock searches for components of the source RPM.

Once the build is ready you should see a number of log files and the source RPM package in the work sub-directory.

Building the RPM packages

Change to the work sub-directory and enter the following commands:

Note that the name of the source RPM might be different. Each time Mock will ask for the root password. Read the Mock documentation to disable password request.

1. To initialize a clean build environment, enter:

mock -v -r epel-7-x86_64 init2. To install the build dependencies, enter:

mock -v -r epel-7-x86_64 install ../dep/*

3. To build RPM packages from the source RPM (note the --no-clean option, this makes sure that the previously prepared build environment is not reset), enter:

mock -v -r epel-7-x86_64 --rebuild --no-clean syslog-ng-3.13.2-3.el7.centos.src.rpm

Once the compilation is ready, Mock will display the location of the resulting RPM packages on the screen:

INFO: Results and/or logs in: /var/lib/mock/epel-7-x86_64/result

Warning: Make sure to copy these files to another, safe directory, because these generated files will be deleted on the next run of Mock.

To test the results and make sure that the fix for your issue is working as expected, install the RPM packages that you have built.

What is next?

In an ideal case you have built the package successfully for the first time, installed the results and reported back that your issue has been solved. The fix will be part of the next release.

Once you have a working package you should perform a little cleanup on your machine. It can free gigabytes of storage, especially if you build packages for multiple distributions and releases. To clean up the files that are not required anymore, enter:

mock –scrub=all

After this, you can also remove the related Docker images.