Though syslog protocol is very common, some customers are experiencing challenges to set it up correctly. This blog article will provide some best practices and guidance through its syslog proxy installation process, including its configuration into LogPoint.
A syslog server can be easily set-up to forward logs, but a very basic configuration will not propagate the source IP of the device to LogPoint. As a result, some detections rules, dashboard and reports might be broken into LogPoint.
To workaround this issue, it is required to implement a proper Syslog Proxy server, so that device’s source IP address is properly relayed to LogPoint server.
For the purpose of this article we will use rsyslog that is part of most Linux distributions. The configuration steps can be done on other syslog servers (like syslog-ng or NxLog Enterprise Edition) but might require a deep review of their technical documentation.v
The example above shows the challenge to work with a syslog relay server that has not been properly configured. The syslog sources are seen through a single very active IP address (syslog server). This leads to many challenges:
A single processing policy has to manage all the data sources
Normalisation rules management becomes complex. Many heterogeneous parsers have to be enabled while some of them might be conflicting.
Routing policy is tricky to configure, especially when data has to be routed to different repositories and accommodate different retention policies.
LogPoint license usage can’t be monitored easily
Some dashboards, reports and detection rules might not work out-of-the-box and therefore require customisation and fix.
There are several options to keep the source IP of the device:
implement UDP or TCP syslog relay (Legacy or IETF format)
implement syslog UDP spoofing on rsyslog
In this diagram, the syslog server has been properly set-up to keep the source IP address of each devices along the syslog server IP. This allows to use a specific processing policy per data-sources, once configured the syslog proxy is transparent to LogPoint.
The purpose of Rsyslog’s UDP spoof is to replace the original source IP address (rsyslog) by the original device IP address.
There are many benefits to this approach:
The syslog proxy is fully transparent and doesn’t require any specific configuration into LogPoint
UDP spoof allows very high throughput
It is quite easy to setup
But also drawbacks:
Downstream data-loss can occur due to use of UDP protocol
UDP spoofing can be detected as a threat by IDS/IPS or NDR devices
UDP spoofing doesn’t work well on routes that uses IP Masquerade or Network address translation.
The first configuration step is to make sure that the rsyslog server is able to receive upstream logs through UDP or TCP.
A simple way to achieve this is to make sure the following lines are present and commented out into /etc/rsyslog.conf :
Before proceding you have to make sure that using Syslog UDP is not a problem for your customer or organization. UDP is known to be less reliable and secure than TCP –
The first step is to check that the syslog proxy server have rsyslog and omudpspoof modules installed. If not, this can be fixed with the command below:
Not all the Linux distributions provides omudpspoof module.
For example, at the time I’m writing this blog, Fedora and Redhat do, whereas Ubuntu doesn’t.
If you can’t change the operating system, it is still possible to download rsyslog sources and rebuid it from scratch. You have to keep in mind this will require efforts to install and maintain it. I would recommend to choose the right linux distribution if possible.
Enabling UDP spoof forwarding on the syslog proxy server is straightforward, you just need to create a configuration file under /etc/rsyslog.d with the following contents :
/etc/rsyslog.d/99-fwlpspoof.conf
Restart the rsyslog service for this configuration to take effect.
As mentioned previously, no specific configuration is required on the LogPoint server, appart from the usual device, fetcher and processing policy declarations.
TCP syslog relay offers a much versatile and reliable alternative to UDP spoof. There is no constraints on the forwarding protocol and relayed syslog can cop with IP Masquerading and Network address translation as the source device IP assignment is managed at destination by LogPoint.
The first configuration step is to make sure that the rsyslog server is able to receive upstream logs through UDP or TCP.
A simple way to achieve this is to make sure the following lines are present and commented out into /etc/rsyslog.conf :
/etc/rsyslog.d/99-fwlpudp.conf
/etc/rsyslog.d/99-fwlptcp.conf
Depending on the requirement, different syslog format can be considered when forwarding syslogs – the details below have been extracted from RSYSLOG online documentation (https://www.rsyslog.com/doc/v8-stable/configuration/templates.html):
The configuration is quite straightforward. Start by declaring the syslog proxy device and make sure the Time Zone is consistent with the device OS configuration.
On the next step, choose Syslog Collector
Finally check option “Use as Proxy” and submit.
Now that the syslog proxy device is set up, we can declare the devices that will send their logs through it.
For each devices, perform the following steps:
Create a device
Define Name and IP Address(es)
Check that the Time Zone is consistant with device OS configuration
4. Choose Syslog Collector
5. Choose applicable Processing Policy and check Uses Proxy option. Then choose the Proxy Server IP and the Hostname of the device being declared.
Et voila! the logs should be reported with the right device IP and timestamp on the search results.
On the example below logger209 is on CET whereas logger210 uses UTC.
