Insider 2019-09: syslog-ng basics; relays; NGINX; Tic-Tac-Toe; sudo; Elastic stack 7; GitHub;

12 Sep 2019

Insider 2019-09: syslog-ng basics; relays; NGINX; Tic-Tac-Toe; sudo; Elastic stack 7; GitHub;

Dear syslog-ng users,

This is the 75th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news.

NEWS

Building blocks of syslog-ng

Recently I gave a syslog-ng introductory workshop at Pass the SALT conference in Lille, France. I got a lot of positive feedback, so I decided to turn all that content into a blog post. Naturally, I shortened and simplified it, but still managed to get enough material for multiple blog posts.

This one gives you an overview of syslog-ng, its major features and an introduction to its configuration.

https://www.syslog-ng.com/community/b/blog/posts/building-blocks-of-syslog-ng

What syslog-ng relays are good for

While there are some users who run syslog-ng as a stand-alone application, the main strength of syslog-ng is central log collection. In this case the central syslog-ng instance is called the server, while the instances sending log messages to the central server are called the clients. There is a (somewhat lesser known) third type of instance called the relay, too. The relay collects log messages via the network and forwards them to one or more remote destinations after processing (but without writing them onto the disk for storage). A relay can be used for many different use cases. We will discuss a few typical examples below.

https://www.syslog-ng.com/community/b/blog/posts/what-syslog-ng-relays-are-good-for

Visualizing NGINX or Apache access logs in Kibana

This tutorial shows you how to parse NGINX or Apache access logs with syslog-ng and create ECS compatible data in Elasticsearch.

https://balagetech.com/visualizing-nginx-access-logs-kibana/

syslog-ng Tic-Tac-Toe

You can play the game of Tic-Tac-Toe using syslog-ng 3.22.1 or later. Learn how to configure syslog-ng for that:

https://kokan.github.io/2019/06/16/syslog-ng-tic-tac-toe.html

Alerting on sudo events using syslog-ng

Why use syslog-ng to alert on sudo events? At the moment, alerting in sudo is limited to E-mail. Using syslog-ng, however, you can send alerts (more precisely, selected logs) to a wide variety of destinations. Logs from sudo are automatically parsed by recent (3.13+) syslog-ng releases, enabling fine-grained alerting. There is a lot of hype around our new Slack destination, so that is what I’ll show here. Naturally, there are many others available as well, including Telegram and, of course, good old E-mail. If something is not yet directly supported by syslog-ng, you can often utilize an HTTP API or write some glue code in Python.

From this blog post you can learn how to build up a syslog-ng configuration step by step and how to use different filters to make sure that you only receive logs (i.e. alerts) that are truly relevant for you.

https://www.syslog-ng.com/community/b/blog/posts/alerting-on-sudo-events-using-syslog-ng

GitHub and syslog-ng

As many of you know, the source code of syslog-ng is available on GitHub, just like its issue tracker. We just learned that GitHub itself is running syslog-ng as part of its stack: https://help.github.com/en/enterprise/2.18/admin/installation/log-forwarding

syslog-ng with Elastic Stack 7

For many years, anything I wrote about syslog-ng and Elasticsearch was valid for all available versions. Well, not anymore. With version 7 of Elasticsearch, there are some breaking changes. These changes are mostly related to the fact that Elastic is phasing out type support. This effects mapping (as the _default_ keyword is not used any more) and the syslog-ng configuration as well (as even though type() is a mandatory parameter, you should leave it empty).

This blog post is a rewrite of one of my earlier blog posts (about creating a heat map using syslog-ng + Elasticsearch + Kibana), focusing on the changes and the new elasticsearch-http() destination:

https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-with-elastic-stack-7