Send your log messages to Slack

One of the many ways members of a team collaborate is to use Slack. From syslog-ng version 3.19, you can send log messages to Slack. You can receive critical log messages in real-time in your Slack client on your mobile or desktop. Learn how to set up alerting to Slack in just a few minutes.

Prerequisites

• At least syslog-ng version 3.19: If your platform carries an earlier version, check https://syslog-ng.com/3rd-party-binaries for pointers to up-to-date packages.

• JSON and HTTP support enabled in syslog-ng: Often it is not part of the main syslog-ng package, but available in sub-packages, like syslog-ng-http or syslog-ng-curl.

• SCL enabled in syslog-ng: The syslog-ng Slack destination is a ready-to-use configuration block in the syslog-ng configuration library (SCL). Make sure that it is enabled in your syslog-ng.conf (it is there in the default configuration):

@include "scl.conf"

• A Slack workspace and a channel in that workspace. If you do not already have them configured, do it before you continue.

Once you have all the prerequisites in place you can start configuring them.

Configuring Slack

1. Login to your workspace.

2. Navigate to https://api.slack.com/..

3. You will first create an application. Click Start building.

4. In the pop-up window, enter the name of your application into the App Name field, for example, “my syslog-ng robot”.

5. Select the Development Slack Workspace where you want to use the App.

6. Click Create App. It will display the Basic Information overview page for your freshly created App.

7. Click Incoming Webhooks.

8. Make sure that the Activate Incoming Webhooks is set to On.

9. Click Add New Webhook to Worspace.

10. From the Post to list, select the channel or user who will receive the syslog-ng messages through the Webhook.

11. Click Authorize.

You should now see a new line in the Webhook URL section. Leave this browser window open, because you will need that URL soon, while configuring syslog-ng.

For a quick test you can even use the on-screen command line example to send a message to Slack using the freshly created Webhook.

Configuring syslog-ng

The following is the simplest configuration that you can use to send your log messages to Slack. Append this to your syslog-ng.conf file or save it to a separate file under the /etc/syslog-ng/conf.d/ directory (if your configuration allows doing that).

destination d_slack {
  slack(
    hook-url("https://hooks.slack.com/services/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")
  );
};

log {source(src); destination(d_slack); };

Replace the value of “hook-url” with the value of the Webhook URL you have created in the last step of the Slack configuration process. You might also have to replace the name of the source in the log statement with your actual source name. The default source name for local logs on openSUSE / SLES is “src.

Testing

Because there are no filters in the current syslog-ng configuration, all local log messages will be pushed to Slack. You will most likely see some incoming messages in your channel even before sending a test message. Because there is a rate limit on incoming messages (one message per second), syslog-ng throttles messages. Therefore the messages might be displayed with a slight delay.

If there are no incoming messages, you can generate your own:

logger "this is a test message"

You should see the test message in your Slack client within a few seconds.

Customizing

Once you have tested that your Slack destination works, you can further customize it. There are various possibilities for customization. The following is just a very simple example, adding a few extra pieces of information about the sender. In the example, I point you to the syslog-ng Twitter account, but it could be your company knowledge base, or any other webpage providing useful information about the log messages.

destination d_slack {
  slack(
    hook-url("https://hooks.slack.com/services/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")
    author-name("syslog-ng test robot")
    author-link("https://twitter.com/sngose")
    author-icon("https://pbs.twimg.com/profile_images/434292432335695872/NP-5Ip4p_400x400.png")
  );
};

There are many more options. For a complete list, check the documentation at https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.19/administration-guide/44#TOPIC-1095150.

Obviously, you do not want to send all your log messages to a Slack channel. Due to the rate limit, it would be slow, and it would not function as an alert any more. You can use parsers and filters to make sure that only relevant messages reach the Slack channel. You can learn more about parsing and filtering from my Suricata blog at https://www.syslog-ng.com/community/b/blog/posts/analyze-your-suricata-logs-in-real-time-using-syslog-ng, or by reading the documentation at https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition/3.19.

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or you can even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/balabit/syslog-ng. On Twitter, I am available as @PCzanik.