syslog-ng documentation

Your main source of knowledge

The syslog-ng product family has an extensive documentation, covering everything from how to install a product to the most complex configuration and settings descriptions. If you cannot find an answer to your question, try the mailing list - our community is always eager to help.

syslog-ng Premium Edition

Contents

7.4.1. Procedure – Prerequisites

To send messages from syslog-ng PE to HDFS, complete the following steps.

Steps: 

  1. If you want to use the Java-based modules of syslog-ng PE (for example, the Elasticsearch, HDFS, or Kafka destinations), download and install the Java Runtime Environment (JRE), 1.7 (or newer).

    The Java-based modules of syslog-ng PE are tested and supported when using the Oracle implementation of Java. Other implementations are untested and unsupported, they may or may not work as expected.

  2. Download the Hadoop Distributed File System (HDFS) libraries (version 2.x) from http://hadoop.apache.org/releases.html.

  3. Extract the HDFS libraries into a temporary directory, then collect the various .jar files into a single directory (for example, /opt/hadoop/lib/) where syslog-ng PE can access them. You must specify this directory in the syslog-ng PE configuration file. The files are located in the various lib directories under the share/ directory of the Hadoop release package. (For example, in Hadoop 2.7, required files are common/hadoop-common-2.7.0.jar, common/libs/*.jar, hdfs/hadoop-hdfs-2.7.0.jar, hdfs/lib/*, but this may change between Hadoop releases, so it is easier to copy every .jar file into a single directory.