Actions

Tags

Peter Czanik
12 Jun 2019
Sending logs to Google Stackdriver using syslog-ng PE

Google Stackdriver collects and analyses logs, events and metrics of your infrastructure. Using syslog-ng PE 7.0.14 or later, you can send your logs to Google Stackdriver. While originally designed to quickly respond to events in the Google Cloud Platform (GCP), you can use Google Stackdriver with any other cloud providers (like Amazon Web Services) or on premises data as well. This way you can view events of a hybrid infrastructure at a single location.

Before you begin

In order to test the Google Stackdriver destination of syslog-ng PE you need two things:

  • a Google account

  • syslog-ng PE 7.0.14 or later

You can test both for free for a limited amount of time. You can learn more about syslog-ng PE and ask for a trial version at https://www.syslog-ng.com/products/log-management-software/

Configuring Google Stackdriver

It is very tempting just to click “Try it free” on the Google Stackdriver page. It most likely works starting from there if you are already a GCP customer. But if you are just trying to push your on-premise or your AWS logs to Google Stackdriver, you better follow the steps outlined in the syslog-ng PE documentation: http://support.oneidentity.com/technical-documents/syslog-ng-premium-edition/7.0.14/administration-guide/sending-and-storing-log-messages-destinations-and-destination-drivers/stackdriver-sending-logs-to-the-google-stackdriver-cloud/configuring-syslog-ng-pe-to-send-logs-to-google-stackdriver

Make sure that you complete all the steps outlined in the documentation. Save the JSON file containing the key for the service account to a location where you can easily find it and note down the project ID.

Configuring syslog-ng PE

If you have not done so yet, install syslog-ng PE in server mode. For that you need a valid (trial) license. Once syslog-ng is up and running locally you can add the Stackdriver destination.

First of all, copy the downloaded JSON file to the location of your syslog-ng configuration, the /opt/syslog-ng/etc/ directory.

Next append a few lines to the syslog-ng PE configuration in /opt/syslog-ng/etc/syslog-ng.conf After installation it provides you with a working minimal configuration. It collects local log messages and saves them to /var/log/messages. The name of the source it creates is s_local. We reuse it for the Stackdriver destination in the log statement:

destination d_stackdriver {
  stackdriver(
    gcp_auth_header(
      credentials("/opt/syslog-ng/etc/czpsngstackdriver-01fcc6750db7.json")
    )
    log_id("mylogid")
    resource(
      generic_node(
        project_id("czpsngstackdriver")
        location("EU/Budapest")
        namespace("my cluster")
        node_id("$HOST")
      )
    )
  );
};

log {source(s_local); destination(d_stackdriver);};

As you can see numerous time in the documentation, while most of syslog-ng is using the hyphen (-) and underscore (_) characters interchangeably, in the Stackdriver destination driver you have to use the underscore.

For credentials() you need to provide the location of the JSON file. For project_id use the name you entered on the web interface.

Read the Google Stackdriver documentation to learn more about the other fields in the configuration. It is available at https://cloud.google.com/monitoring/api/resources

Testing

Once you have everything configured, reload syslog-ng so the new configuration comes into effect. In a short while your log messages should be arriving to Google Stackdriver. On the Dashboard on the left hand side click on Logging. This will open up a new tab in your browser.

Note that when you first open this page, you will see the service account logs. On the top left of the screen in the drop down menu switch to Generic Node. You should see your Linux logs on the screen.

Uploading name-value pairs

The default template sends the basic syslog fields in JSON format to Stackdriver. You can extend your configuration to send name-value pairs created by syslog-ng as well. For example, recent versions of syslog-ng parse sudo log messages automatically and create name-value pairs out of them. These start with .sudo, where the leading dot is turned into an underscore when formatted as JSON.

Here is the line you need to add to your configuration (for example right under log_id()):

    json-payload("$(format-json --scope rfc5424 --scope dot-nv-pairs --exclude DATE --key ISODATE)")

Check the value-pairs documentation for a detailed description of possible configuration options for the JSON template.

Testing name-value pairs

The easiest way is enter a few commands through sudo and then search the results in the Stackdriver web interface. You can search the content of the JSON fields easily as the search interface helps you to find the names of the fields. Here I searched for sudo logs coming from user czanik:

Stackdriver -- sudo

Learn more

If you want to learn more about how to use syslog-ng PE with the Google Stackdriver join our next webinar: https://www.syslog-ng.com/event/live-webinar-how-to-use-the-syslogng-pes-new-google-stackdriver-destin8139195/

If you’d like to try sending logs with syslog-ng PE to Google Stackdriver, download a trial version of syslog-ng Premium Edition.

If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or you can even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/balabit/syslog-ng. On Twitter, I am available as @PCzanik.

Related Content