Actions

Tags

Peter Czanik
20 Apr 2021
First steps of sending alerts to Discord and others from syslog-ng: http() and Apprise

A returning question I get is: “I see, that you can send alerts from syslog-ng to Slack and Telegram, but do you happen to support XYZ?” Replace XYZ with Discord and countless others. Up until recently, my regular answer has been: “Take a look at the Slack destination of syslog-ng, and based on that, you can add support for your favorite service”. Then I learned about Apprise, a notification library for Python, supporting dozens of different services.

This blog is the first part of a series. It covers how to send log messages to Discord using the http() destination of syslog-ng and an initial try at using Apprise for alerting.

The next part will show you a lot more flexible version of the Apprise integration: making fields configurable using templates and using a block to hide implementation details from the user.

The Python code in these two blogs is sample code, provided to you for inspiration. They are not for production use, as among others, they lack error handling and reporting. If time and my Python knowledge permits, I might have a more production-ready code later on, that I plan to cover in a third blog.

Before you begin

The http() destination is enabled by default on FreeBSD, but it needs to be installed from sub-packages on most Linux distributions. It is called syslog-ng-curl on SLES/openSUSE, syslog-ng-http on RHEL/CentOS/Fedora, and syslog-ng-mod-http on Debian/Ubuntu. To use the Python destination, you need to compile syslog-ng yourself on FreeBSD. On RPM distros, install the syslog-ng-python package and on Debian/Ubuntu, syslog-ng-mod-python.

When I wrote this blog, I used the latest syslog-ng version available, which is 3.31.2. Even though any recent syslog-ng version should work from the past two years, so from about 3.20 and probably even earlier,still, I’d recommend using the latest version. Check https://www.syslog-ng.com/products/open-source-log-management/3rd-party-binaries.aspx if up-to-date 3rd-party packages are available for your operating system.

Python() vs. http()

The http() destination was with us for many years now, just as the python() destination. Both improved in many ways over the years. And both have their own strengths and weaknesses.

The strength of the http() destination is that it is both fast and lightweight. However, figuring out how to implement sending logs to a given service might prove difficult, and often impossible if it requires more than just dumping the logs in the required format. And you need to figure out each service separately.

The Python() destination is considerably slower than using http() and it requires more resources. Many services have Python libraries to access them, saving you time figuring out all the details of how a protocol works and even enabling you to access services that http() cannot support. But you still needed to figure out how to support each of them separately.

This is where Apprise changes the equation. It is a notification library for Python supporting dozens of different services. Some are already supported by syslog-ng through the http() destination, like Telegram or Slack, but most of them I have never heard of before. And while Python still uses more resources than an http() destination, sending occasional alerts does not require the high performance of the http() destination. Using Apprise can save you a lot of documentation reading and trial and error.

Creating a Discord destination based on http()

Unlike many other services, Discord is easy both on the service and on the syslog-ng side. It took barely five minutes to configure everything, including reading the documentation and testing. In the channel where you want to send your alerts, click “Edit channel”. Then, choose “Integrations”, “Webhooks”, and finally “New Webhook”. You can set a name here, but what is more important: copy the URL to the webhook.

I used two documents to configure syslog-ng:

The first gives you a sample syslog-ng configuration to use the http() destination and describes the options. The second one shows the parameters you need to set to send data to a Discord webhook. I combined the two into a simple configuration, saved under /etc/syslog-ng/conf.d/discrod.conf and I was ready to go:

source s_net {
  tcp(port(514) flags(syslog-protocol));
};

destination d_http {
    http(
        url("https://discord.com/api/webhooks/xxx/yyy")
        method("POST")
        user-agent("syslog-ng User Agent")
        headers("Content-Type: application/json")
        body('{"username": "test", "content": "${ISODATE} ${MESSAGE}"}')
    );
};

log {
    source(s_net);
    destination(d_http);
};

As I do not want my Discord channel flooded with log messages (as was the case after one of my earlier sample configuration :-) ), I anonymized the webhook URL. I used a network source instead of using system logs to make sure that I do not overflow Discord with log messages banning me from the service.

Testing is as easy as:

logger -n 127.0.0.1 -T -P 514 This is a test

And the log message should appear on your Discord channel with the username “test”, the current date and the test message.

Sending logs to Discord using the http() destination is easy, but it is more of an exception than a rule.

Creating a Discord destination based on Apprise

Before using Apprise, you have to install it. On Fedora, it is easy, as it is a part of the distribution:

dnf install apprise

Installation on other operating systems is similarly easy. The major difference is that when you install a Python module using pip, it is not tracked by the package manager of the operating system:

pip3 install apprise

Just as before, save the following configuration with a .conf extension under /etc/syslog-ng/conf.d/ (or append to syslog-ng.conf, if your distribution of choice does not set up such a directory).

source s_net {
  tcp(port(514) flags(syslog-protocol));
};
destination d_python_apprise {
    python(
        class("AppRise")
        value-pairs(scope(rfc5424))
        options(url "https://discord.com/api/webhooks/xxx/yyy")
    );
};
log {
    source(s_net);
    destination(d_python_apprise);
};
python {
import apprise
class AppRise(object):
    def init(self, options):
#        print(options)
        self.apobj = apprise.Apprise()
        self.apobj.add(options["url"])
        return True
    def send(self, msg):
#        print(msg['MESSAGE'].decode('utf-8'))
        self.apobj.notify(
            body=msg['MESSAGE'].decode('utf-8'),
            title=msg['DATE'].decode('utf-8'),
        )
        return True
};

As you can see, we start the configuration with a network source again. It means that testing the new Python-based destination will be the same as for the http()-based one.

Python destinations in syslog-ng have two parts: you have a configuration part, just like any other destination,and there is also a code part. The Python code can be included in the syslog-ng configuration or stored in an external file. Here we embed it in the configuration. In the Python destination, there is a single mandatory setting: the class name. This class in the Python code has the method to send the logs and optionally others. The valule-pairs() options specifies for the Python() destination which name-value pairs to forward to the Python code and make available in dict format. Finally, options can pass options from the syslog-ng configuration to the Python code. In this case the URL is passed as an option.

The last block in the configuration is the Python code. It starts by importing the Apprise module. Then a class is defined. Make sure that the name here and in the configuration part matches.

The init() method runs only once, when the Python code is loaded. Here we initialize Apprise and add the URL to it.

The send() method runs each time a message is sent. The log message is passed on to this method as a dict. Apprise sends two blocks of data to notification services, title and body. How these appear is up to the given service. In case of email, the “title” variable becomes the subject of the email, while the content of the “body” variable will be the body of the email. In case of Discord, the “title” becomes the first line of the message, and the body will be the rest of the message. Here we can refer to single fields, or macros in syslog-ng terms, as title or body. And these are hardcoded into the Python code.

What is next?

The Python code and configuration above is good enough to get started. But it is far from being elegant or flexible. Implementation details are exposed on the configuration side. And data sent is hardcoded in the Python code. On the positive side even this code can get you started sending log messages to dozens of different services, not just Discord as the http()-based destination in this blog. Just replace the URL with something else supported by Apprise. For a complete list, check the documentation in the Apprise GitHub repository at https://github.com/caronc/apprise which also helps to set up the service side of the project.

In the next blog, we will keep working with Apprise. I will introduce you to using templates in the Python destination and will hide implementation details by using a configuration block.

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @Pczanik.

Related Content