Version 4.6 of syslog-ng introduced windows-eventlog-xml-parser(), a dedicated parser for XML-formatted event logs from Windows. It makes the EventData portion of log messages more useful, as it combines two arrays into a list of name-value pairs.
Before you begin
To use the windows-eventlog-xml-parser(), you need at least syslog-ng version 4.6. It is already available on some cutting-edge Linux distributions and FreeBSD ports, but by the time you are reading this, you have to compile it yourself or use a third-party repository in most cases. Check https://syslog-ng.org/3rd-party-binaries/ if there are ready-to-use 3rd party binaries available for your OS.
You also need some XML formatted Windows eventlogs. I used syslog-ng PE Windows Agent, which is a commercial product by One Identity, but it was also tested using syslog-ng PE Windows Event Collector. Other log collectors on Windows might also work, but they have not been tested yet.
Configuring the Windows Agent
Open the syslog-ng Windows Agent configuration interface, and configure the destination. Enter the IP address of your server and switch to the Messages tab of the window. Replace the message template with the following:
${EVENT_MSG_XML}
This ensures that Windows events are sent in XML format by the syslog-ng Windows Agent.
Configuring syslog-ng
Create a new configuration snippet under /etc/syslog-ng/conf.d/ if your Linux distribution supports it, or append syslog-ng.conf with the following content:
config:
source s_net {
syslog(port(601));
};
parser p_plainxml {
xml(prefix('plainxml.'));
};
parser p_winxml {
windows-eventlog-xml-parser(prefix('winxml.'));
};
destination d_fromnet_p {
file("/var/log/fromnet_p.json" template("$(format-json --scope rfc5424 --scope dot-nv-pairs
--rekey .* --shift 1 --scope nv-pairs)\n") );
};
destination d_fromnet_w {
file("/var/log/fromnet_w.json" template("$(format-json --scope rfc5424 --scope dot-nv-pairs
--rekey .* --shift 1 --scope nv-pairs)\n") );
};
log {source(s_net); parser(p_plainxml); destination(d_fromnet_p);};
log {source(s_net); parser(p_winxml); destination(d_fromnet_w);};
This adds a network source using the RFC5424 syslog protocol on port 601. Make sure that this port is not blocked by your firewall or by SELinux. You can also use a different one, just make sure that you configure the same port in the syslog-ng Windows Agent.
All other configuration elements are doubled. One set is for the traditional xml() parser, while the other one is for the windows-eventlog-xml-parser(). Logs are saved into JSON formatted files, as this way you can see all parsed name-value pairs created from the log messages.
Testing
After you have configured both the client and the server, you should see incoming logs in /var/log/fromnet_*. You can see two sample log messages below. Note that these log messages are large, over a page long, even if they are just a single line.
The first log message was parsed by the xml() parser:
{"plainxml":{"Event":{"_xmlns":"http://schemas.microsoft.com/win/2004/08/events/event","System":{"Version":"0","TimeCreated":{"_SystemTime":"2024-02-16T10:33:43.6311232Z"},"Task":"13824","Provider":{"_Name":"Microsoft-Windows-Security-Auditing","_Guid":"{54849625-5478-4994-a5ba-3e3b0328c30d}"},"Opcode":"0","Level":"0","Keywords":"0x8020000000000000","Execution":{"_ThreadIDD":"940","_ProcessID":"892"},"EventRecordID":"60986","EventID":"5379","Correlation":{"_ActivityID":"{81f1fbd6-60a9-0001-8d03-f281a960da01}"},"Computer":"czp-win11vm","Channel":"Security"},"RenderingInfo":{"_Culture":"en-GB","Task":"\"Security Group Management\"","Provider":"\"Microsoft Windows security auditing.\"","Opcode":"Info","Message":"\"Credential Manager credentials were read.\\r\\n\\r\\nSubject:\\r\\n\\tSecurity ID:\\t\\tS-1-5-21-2375527766-1973386784-3876262867-1000\\r\\n\\tAccount Name:\\t\\tPeter Czanik\\r\\n\\tAccount Domain:\\t\\tCZP-WIN11VM\\r\\n\\tLogon ID:\\t\\t0x9E9984\\r\\n\\tRead Operation:\\t\\tRead Credential\\r\\n\\r\\nThis event occurs when a user performs a read operation on stored credentials in Credential Manager.\"","Level":"Information","Keywords":{"Keyword":"\"Audit Success\""},"Channel":"Security"},"EventData":{"Data":{"_Name":["SubjectUserSid","SubjectUserName","SubjectDomainName","SubjectLogonId","TargetName","Type","CountOfCredentialsReturned","ReadOperation","ReturnCode","ProcessCreationTime","ClientProcessId"],"Data":["S-1-5-21-2375527766-1973386784-3876262867-1000","Peter Czanik","CZP-WIN11VM","0x9e9984","XboxLive","1","1","%%8099","3221226021","2024-02-16T10:33:43.2876190Z","29548"]}}}},"TRANSPORT":"rfc6587","SOURCE":"s_net","SDATA":{"win@18372.4":{"EVENT_USERNAME":"CZP-WIN11VM\\Peter Czanik","EVENT_TYPE":"Success Audit","EVENT_TASK":"Security Group Management","EVENT_SOURCE":"Microsoft Windows security auditing.","EVENT_SID":"N/A","EVENT_REC_NUM":"60986","EVENT_PROVIDER":"Microsoft-Windows-Security-Auditing","EVENT_NAME":"Security","EVENT_LEVEL":"0","EVENT_ID":"5379","EVENT_HOST":"czp-win11vm","EVENT_FACILITY":"16","EVENT_CATEGORY":"Security Group Management"},"meta":{"sysUpTime":"93040","sequenceId":"211"}},"PROGRAM":"Microsoft_Windows_security_auditing.","PRIORITY":"notice","PID":"27192","MESSAGE":"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/><EventID>5379</EventID><Version>0</Version><Level>0</Level><Task>13824</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2024-02-16T10:33:43.6311232Z'/><EventRecordID>60986</EventRecordID><Correlation ActivityID='{81f1fbd6-60a9-0001-8d03-f281a960da01}'/><Execution ProcessID='892' ThreadID='940'/><Channel>Security</Channel><Computer>czp-win11vm</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>S-1-5-21-2375527766-1973386784-3876262867-1000</Data><Data Name='SubjectUserName'>Peter Czanik</Data><Data Name='SubjectDomainName'>CZP-WIN11VM</Data><Data Name='SubjectLogonId'>0x9e9984</Data><Data Name='TargetName'>XboxLive</Data><Data Name='Type'>1</Data><Data Name='CountOfCredentialsReturned'>1</Data><Data Name='ReadOperation'>%%8099</Data><Data Name='ReturnCode'>3221226021</Data><Data Name='ProcessCreationTime'>2024-02-16T10:33:43.2876190Z</Data><Data Name='ClientProcessId'>29548</Data></EventData><RenderingInfo Culture='en-GB'><Message>Credential Manager credentials were read. Subject: 	Security ID:		S-1-5-21-2375527766-1973386784-3876262867-1000 	Account Name:		Peter Czanik 	Account Domain:		CZP-WIN11VM 	Logon ID:		0x9E9984 	Read Operation:		Read Credential This event occurs when a user performs a read operation on stored credentials in Credential Manager.</Message><Level>Information</Level><Task>Security Group Management</Task><Opcode>Info</Opcode><Channel>Security</Channel><Provider>Microsoft Windows security auditing.</Provider><Keywords><Keyword>Audit Success</Keyword></Keywords></RenderingInfo></Event>","HOST_FROM":"172.16.167.128","HOST":"czp-win11vm","FACILITY":"local0","DATE":"Feb 16 11:33:43"}
The second log message was originally the same, but parsed by the windows-eventlog-xml-parser():
{"winxml":{"Event":{"_xmlns":"http://schemas.microsoft.com/win/2004/08/events/event","System":{"Version":"0","TimeCreated":{"_SystemTime":"2024-02-16T10:33:43.6311232Z"},"Task":"13824","Provider":{"_Name":"Microsoft-Windows-Security-Auditing","_Guid":"{54849625-5478-4994-a5ba-3e3b0328c30d}"},"Opcode":"0","Level":"0","Keywords":"0x8020000000000000","Execution":{"_ThreadIDD":"940","_ProcessID":"892"},"EventRecordID":"60986","EventID":"5379","Correlation":{"_ActivityID":"{81f1fbd6-60a9-0001-8d03-f281a960da01}"},"Computer":"czp-win11vm","Channel":"Security"},"RenderingInfo":{"_Culture":"en-GB","Task":"\"Security Group Management\"","Provider":"\"Microsoft Windows security auditing.\"","Opcode":"Info","Message":"\"Credential Manager credentials were read.\\r\\n\\r\\nSubject:\\r\\n\\tSecurity ID:\\t\\tS-1-5-21-2375527766-1973386784-3876262867-1000\\r\\n\\tAccount Name:\\t\\tPeter Czanik\\r\\n\\tAccount Domain:\\t\\tCZP-WIN11VM\\r\\n\\tLogon ID:\\t\\t0x9E9984\\r\\n\\tRead Operation:\\t\\tRead Credential\\r\\n\\r\\nThis event occurs when a user performs a read operation on stored credentials in Credential Manager.\"","Level":"Information","Keywords":{"Keyword":"\"Audit Success\""},"Channel":"Security"},"EventData":{"Data":{"Type":"1","TargetName":"XboxLive","SubjectUserSid":"S-1-5-21-2375527766-1973386784-3876262867-1000","SubjectUserName":"Peter Czanik","SubjectLogonId":"0x9e9984","SubjectDomainName":"CZP-WIN11VM","ReturnCode":"3221226021","ReadOperation":"%%8099","ProcessCreationTime":"2024-02-16T10:33:43.2876190Z","CountOfCredentialsReturned":"1","ClientProcessId":"29548"}}}},"TRANSPORT":"rfc6587","SOURCE":"s_net","SDATA":{"win@18372.4":{"EVENT_USERNAME":"CZP-WIN11VM\\Peter Czanik","EVENT_TYPE":"Success Audit","EVENT_TASK":"Security Group Management","EVENT_SOURCE":"Microsoft Windows security auditing.","EVENT_SID":"N/A","EVENT_REC_NUM":"60986","EVENT_PROVIDER":"Microsoft-Windows-Security-Auditing","EVENT_NAME":"Security","EVENT_LEVEL":"0","EVENT_ID":"5379","EVENT_HOST":"czp-win11vm","EVENT_FACILITY":"16","EVENT_CATEGORY":"Security Group Management"},"meta":{"sysUpTime":"93040","sequenceId":"211"}},"PROGRAM":"Microsoft_Windows_security_auditing.","PRIORITY":"notice","PID":"27192","MESSAGE":"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/><EventID>5379</EventID><Version>0</Version><Level>0</Level><Task>13824</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2024-02-16T10:33:43.6311232Z'/><EventRecordID>60986</EventRecordID><Correlation ActivityID='{81f1fbd6-60a9-0001-8d03-f281a960da01}'/><Execution ProcessID='892' ThreadID='940'/><Channel>Security</Channel><Computer>czp-win11vm</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>S-1-5-21-2375527766-1973386784-3876262867-1000</Data><Data Name='SubjectUserName'>Peter Czanik</Data><Data Name='SubjectDomainName'>CZP-WIN11VM</Data><Data Name='SubjectLogonId'>0x9e9984</Data><Data Name='TargetName'>XboxLive</Data><Data Name='Type'>1</Data><Data Name='CountOfCredentialsReturned'>1</Data><Data Name='ReadOperation'>%%8099</Data><Data Name='ReturnCode'>3221226021</Data><Data Name='ProcessCreationTime'>2024-02-16T10:33:43.2876190Z</Data><Data Name='ClientProcessId'>29548</Data></EventData><RenderingInfo Culture='en-GB'><Message>Credential Manager credentials were read. Subject: 	Security ID:		S-1-5-21-2375527766-1973386784-3876262867-1000 	Account Name:		Peter Czanik 	Account Domain:		CZP-WIN11VM 	Logon ID:		0x9E9984 	Read Operation:		Read Credential This event occurs when a user performs a read operation on stored credentials in Credential Manager.</Message><Level>Information</Level><Task>Security Group Management</Task><Opcode>Info</Opcode><Channel>Security</Channel><Provider>Microsoft Windows security auditing.</Provider><Keywords><Keyword>Audit Success</Keyword></Keywords></RenderingInfo></Event>","HOST_FROM":"172.16.167.128","HOST":"czp-win11vm","FACILITY":"local0","DATE":"Feb 16 11:33:43"}
Here I reformatted the JSON to make it easier to read, and I only pasted the difference here, which is EventData. As you can see, when using the xml() parser, name and data are two separate arrays. These are difficult to read, especially if you have a longer list:
"EventData": {
"Data": {
"_Name": [
"SubjectUserSid",
"SubjectUserName",
"SubjectDomainName",
"SubjectLogonId",
"TargetName",
"Type",
"CountOfCredentialsReturned",
"ReadOperation",
"ReturnCode",
"ProcessCreationTime",
"ClientProcessId"
],
"Data": [
"S-1-5-21-2375527766-1973386784-3876262867-1000",
"Peter Czanik",
"CZP-WIN11VM",
"0x9e9984",
"XboxLive",
"1",
"1",
"%%8099",
"3221226021",
"2024-02-16T10:33:43.2876190Z",
"29548"
]
}
}
Using the windows-eventlog-xml-parser() these are put together into easy-to-read name-value pairs:
"EventData": {
"Data": {
"Type": "1",
"TargetName": "XboxLive",
"SubjectUserSid": "S-1-5-21-2375527766-1973386784-3876262867-1000",
"SubjectUserName": "Peter Czanik",
"SubjectLogonId": "0x9e9984",
"SubjectDomainName": "CZP-WIN11VM",
"ReturnCode": "3221226021",
"ReadOperation": "%%8099",
"ProcessCreationTime": "2024-02-16T10:33:43.2876190Z",
"CountOfCredentialsReturned": "1",
"ClientProcessId": "29548"
}
}
What is next?
Using the windows-eventlog-xml-parser(), EventData is not just easier to read, but also easier to use. The name-value pairs are ready to be used and do not require any further processing. You can now easily create filters based on information in EventData, or store selected name-value pairs in databases.
-
If you have questions or comments related to syslog-ng, do not hesitate to contact us. You can reach us by email or even chat with us. For a list of possibilities, check our GitHub page under the “Community” section at https://github.com/syslog-ng/syslog-ng. On Twitter, I am available as @PCzanik, on Mastodon as @Pczanik@fosstodon.org.