Sumo Logic is one of the most popular cloud-based log management and security analytics services. They provide their own log shippers, but also work with others, including syslog-ng. The use of syslog-ng is well documented on their support website, but the actual configuration part can easily be simplified. A new, dedicated Sumo Logic destination is already available in syslog-ng Git sources and will be released soon, as part of syslog-ng 3.27. It is based either on the network() or the http() destination of syslog-ng and implemented as an SCL, hiding away most of the configuration details from the users.
Before you begin
Once syslog-ng version 3.27 is out, there will be ready to use packages for the most popular Linux distributions. Check https://syslog-ng.com/3rd-party-binaries for links to package repositories. If you use openSUSE/SLES or Fedora/RHEL, my git snapshot packages also feature the new destination: https://syslog-ng.com/blog/rpm-packages-from-syslog-ng-git-head/ You can also use DBLD to build packages yourself: https://www.syslog-ng.com/community/b/blog/posts/dbld-a-syslog-ng-developer-tool-not-just-for-developers
Most Linux distributions package syslog-ng in a modular way. The base syslog-ng package only contains core features without external dependencies. To get the Sumo Logic destination working, you need two additional modules installed: json and http. Note that JSON support is included in the base package on RPM distributions and, for historical reasons, openSUSE calls the http module as curl.
Configuring Sumo Logic
Obviously, you also need a Sumo Logic account. If you do not have one, it is really easy to register on their website and reach full functionality of Sumo Logic for 30 days. Read their documentation on how to add a new syslog source: https://help.sumologic.com/03Send-Data/Sources/02Sources-for-Hosted-Collectors/Cloud-Syslog-Source The process is very similar for an HTTP source. HTTP is sometimes easier to get through firewalls, but otherwise the log messages, which are sent are the same. In either case, at the end of the source setup, Sumo Logic will provide you with a few values which you will need to use in your syslog-ng configuration.
Configuring syslog-ng
Encryption is mandatory in either case, no matter if you use syslog or HTTP. And that is probably the most difficult part of the client-side configuration. So, as a first step you need to prepare for encryption. Check if you have a directory called /etc/syslog-ng/ca.d and create it if it does not exist. Then change to this directory and setup a DigCert certificate to be used with syslog-ng:
cd /etc/syslog-ng/ca.d wget -O digicert_ca.der https://www.digicert.com/CACerts/DigiCertHighAssuranceEVRootCA.crt openssl x509 -inform der -in digicert_ca.der -out digicert_ca.crt ln -s digicert_ca.crt `openssl x509 -noout -hash -in digicert_ca.crt`.0
When you set up a syslog source in Sumo Logic, at the end it shows you a form with a few values. It has a button to copy values. You can access these values later too, and also generate new tokens if necessary.
In most Linux distributions, syslog-ng is configured in a way that you can include configuration snippets saved to files with a .conf extension under the /etc/syslog-ng/conf.d/ directory (otherwise append the configuration to syslog-ng.conf). Create a new file, for example sumo.conf, start a new line with a # (comment sign) and paste the values from the Sumo Logic web interface. While this step is not strictly necessary, it makes editing easier later. The line will look something similar:
# Token: rqf/bdxYVaBLFMoU39[...]myCCC5jwETm@41123, Host: syslog.collection.eu.sumologic.com, TCP TLS Port: 6514
The above example is a bit shortened for space and privacy. From these you need two information. The “token” and the “deployment”, the two letters in the host name indicating the site where your logs are stored. You can find the actual configuration below, using the sumologic-syslog() destination:
destination d_sumo_syslog {
sumologic-syslog(token("rqf/bdxYVaBLFMoU39[...]CCC5jwETm@41123") deployment("eu")
tls(peer-verify(required-trusted) ca-dir('/etc/syslog-ng/ca.d'))
);
};
Of course, a destination does not work unless it is included in a log path. Suppose that you have a source called “src” (like openSUSE or Turris Omnia), the simplest log path is:
log {
source(src);
destination(d_sumo_syslog);
};
In real-life situations you will most likely use a lot more complex setup with many parsers and filters making sure that only relevant information is forwarded to Sumo Logic.
Configuring a Sumo Logic HTTP destination is similarly easy. Instead of three different values, you get a single URL at the end:
# https://endpoint1.collection.eu.sumologic.com/receiver/v1/http/ZaVnC4dhaV3_[...]UF2D8DRSnHiGKoq9Onvz-XT7RJG2FA6RuyE5z4A==
From this you will use two information: the two-letter abbreviation identifying the deployment, and the last part of the URL for the collector. As previously, I shortened this quite considerably. The actual configuration using the sumologic-http() destination looks like this:
destination d_sumo_http {
sumologic-http(collector("ZaVnC4dhaV3_[...]UF2D8DRSnHiGKoq9Onvz-XT7RJG2FA6RuyE5z4A==") deployment("eu")
tls(peer-verify(yes) ca-dir('/etc/syslog-ng/ca.d'))
);
};
For older syslog-ng releases
If you use an older syslog-ng release and cannot upgrade for various reasons, you can still take advantage of the new destinations. Of course, you still need HTTP and JSON support enabled in syslog-ng. Download the configuration from https://github.com/syslog-ng/syslog-ng/blob/master/scl/sumologic/sumologic.conf and save it under /etc/syslog-ng/conf.d/ (or append to syslog-ng.conf, if your distribution of choice did not enable includes). On Turris Omnia, this directory is called: /etc/syslog-ng.d/
If “syslog-ng -s” gives you an error, you might need to remove the lines starting with “@requires”, as its format changed recently.
Your own sumologic-syslog() or sumologic-http() destination configuration should be appended to the file where you stored the content of the above configuration snippet.
When you upgrade syslog-ng to a release which already includes the Sumo Logic SCL, it will work but you will see a warning about defining a configuration block multiple times. Simply remove the definitions you downloaded manually.
Next steps
As I mentioned earlier, it is very rare that someone sends all log messages to a central log management or log analytics software. So, you will most likely spend some time on setting up parsers and filters to make sure that only relevant log messages are forwarded to Sumo Logic. You can change the template and send logs in JSON format to Sumo Logic. It is automatically parsed on the server side and rendered in an easy to read format. The possibilities are endless.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik.