There are times where you may only be able to send data to a
single syslog receiver, but all is not lost. Fortunately, many syslog receivers
can handle splitting the feed. In this article we will use syslog-ng to show
the steps needed to split a syslog feed (possibly for the purpose of taking a single feed and sending it to two SIEMs).
As a bonus, we will give you the steps to also receive and split a TLS
encrypted syslog feed.
Basic Requirements
Since we are not handling much volume in this example, we
just need minimum requirements:
Our setup
- Amazon EC2 instance
- Ubuntu Server 18.04 LTS (Red Hat is also possible, but requires different commands)
- 1 vCPU, 1GB RAM, 10GB HD
Basic Setup Steps
The syslog-ng package was not available out of the box for
us so we had to run:
sudo apt update
Install and create a
basic configuration file
sudo su -
apt install
syslog-ng
mv syslog-ng.conf
syslog-ng.bak
vim syslog-ng.conf
Note: We used the
same version found in the “Config version” field after running “syslog-ng
--version”
source s_input {
tcp(ip(0.0.0.0) port(10514)); };
destination d_siem{
udp("10.0.0.1" port(7514) template("$MSG") ); };
destination d_siem2{
tcp("10.0.0.2" port(7515) template("$MSG") ); };
log{
source(s_input);
destination(d_siem);
destination(d_siem2);
};
|
Configuration explanation:
- template("$MSG") ensures that syslog-ng does not add another header to the message
- Single input called s_input listening on all interfaces over TCP port 10514
- Two destinations
2) d_siem2 sending to 10.0.0.2 on TCP 515
Validate the
configuration file
Use the following command to validate and troubleshoot
syslog-ng configuration issues:
syslog-ng -s -f
syslog-ng.conf
Start/restart the
Service
Use the following command to check and restart the syslog-ng
service:
service syslog-ng status
service syslog-ng restart
Validate the port is open
Use the following command to validate listening port is
open:
netstat -an | grep
:10514
tcp 0
0 0.0.0.0:10514
0.0.0.0:* LISTEN
Test the Splitting
Since we are splitting TCP syslog in our example, we can use
a web browser to test it:
http://<Your_syslog-ng_server>:10514
Figure 2: Showing port details
Seeing the results in on the final server on UDP 7514 and
TCP 7514
Figure 3: Confirmation that the UDP 7514 split is
working
Figure 4: Confirmation that the TCP 7515 split is
working
Enabling Syslog over TLS on the Input
We mentioned that there would be a bonus, now let’s enable TLS.
Note: This should be a requirement if your data is traversing
a public network such as the Internet.
Creating private
Certificate Authority (CA)
If you don’t have your own CA or choose not to use a public
CA, you will need to create a CA on your syslog-ng server to sign certificates
sudo su –
cd /root
mkdir CA
cd CA
mkdir certs crl
newcerts private
echo "01"
> serial
cp /dev/null
index.txt
cp /etc/ssl/openssl.cnf
.
vi openssl.cnf
Change from:
dir = ./demoCA
to:
dir = .
:wq to save and exit vi
Generate the CA
certificate
openssl req -new
-x509 -keyout private/cakey.pem -out cacert.pem -days 365 -config openssl.cnf
Creating Server Certificate
Request
Whether you use the local CA or an external/public CA, you
will need to generate a server certificate request.
openssl req -nodes
-new -x509 -keyout serverkey.pem -out serverreq.pem -days 365 -config
openssl.cnf
openssl x509
-x509toreq -in serverreq.pem -signkey serverkey.pem -out tmp.pem
Common name should be the IP address or FQDN of your server.
Leave email field blank.
Signing Server
Certificate with CA
Next you can either get the serverreq.pem signed by your
external CA, or sign it locally with the next step:
openssl ca -config
openssl.cnf -policy policy_anything -out servercert.pem -infiles tmp.pem
rm tmp.pem
Either way you should end up with servercert.pem and
serverkey.pem files, which will be used as
the server certificate and key for TLS encryption.
Copying Certificates
to Syslog Directory
mkdir
/etc/syslog-ng/cert.d
mkdir
/etc/syslog-ng/key.d
mkdir
/etc/syslog-ng/ca.d (if using local Certificate Authority)
cp cacert.pem
/etc/syslog-ng/ca.d/ (if using local Certificate Authority)
cp servercert.pem
/etc/syslog-ng/cert.d/
cp serverkey.pem
/etc/syslog-ng/key.d/
cd
/etc/syslog-ng/cert.d/
chown root
servercert.pem
chmod 600
servercert.pem
cd
/etc/syslog-ng/key.d/
chown root
serverkey.pem
chmod 600
serverkey.pem
cd /etc/syslog-ng/ca.d/
(if using local Certificate Authority)
chown root
cacert.pem (if using local Certificate Authority)
chmod 600 cacert.pem
(if using local Certificate Authority)
Modify syslog-ng
configuration file to use TLS
vim syslog-ng.conf
@version: 3.13
source s_input {
tcp(ip(0.0.0.0) port(10514) tls(key_file("/etc/syslog-ng/key.d/serverkey.pem")
cert_file("/etc/syslog-ng/cert.d/servercert.pem")
peer_verify(optional-untrusted))
); };
destination d_siem{
udp("10.0.0.1" port(7514) template("$MSG") ); };
destination d_siem2{
tcp("10.0.0.2" port(7515) template("$MSG") ); };
log{
source(s_input);
destination(d_siem);
destination(d_siem2);
};
|
Validate the
configuration file
Use the following command to validate and troubleshoot
syslog-ng configuration issues:
syslog-ng -s -f syslog-ng.conf
Start/restart the
Service
Use the following command to check and restart the syslog-ng
service:
service syslog-ng status
service syslog-ng restart
Test the Splitting
Since we are splitting TCP over TLS syslog in our example,
we can use a web browser to test it:
https://<Your_syslog-ng_server>:10514
Figure 5: TLS enabled on the Input
Enabling Syslog over TLS on the Input & Output
Let’s go for the double bonus now and enable TLS on the
Input and TCP Output
Note: This should be a requirement if your data is traversing
multiple public networks such as the Internet.
Modify syslog-ng
configuration file to use TLS
vim syslog-ng.conf
@version: 3.13
source s_input {
tcp(ip(0.0.0.0) port(10514) tls(key_file("/etc/syslog-ng/key.d/serverkey.pem")
cert_file("/etc/syslog-ng/cert.d/servercert.pem")
peer_verify(optional-untrusted)) ); };
destination d_siem{
udp("10.0.0.1" port(7514) template("$MSG") ); };
destination d_siem2{
tcp("10.0.0.2" port(7515) tls(key_file("/etc/syslog-ng/key.d/serverkey.pem")
cert_file("/etc/syslog-ng/cert.d/servercert.pem")
peer_verify(optional-untrusted)) template("$MSG") ); };
log{
source(s_input);
destination(d_siem);
destination(d_siem2);
};
|
Validate the
configuration file
Use the following command to validate and troubleshoot
syslog-ng configuration issues:
syslog-ng -s -f
syslog-ng.conf
Start/restart the
Service
Use the following command to check and restart the syslog-ng
service:
service syslog-ng status
service syslog-ng restart
Test the Splitting
Since we are splitting TCP over TLS syslog in our example,
we can use a web browser to test it:
https://<Your_syslog-ng_server>:10514
Figure 6: TLS enabled on input and TCP output
Set Service to Autostart on Boot
If all is well, now we can set the service to autostart at
boot – just in case we experience a reboot.
sudo update-rc.d
syslog-ng defaults
Added Security
Hopefully the source and destination IP addresses are static
and known. This allows us to further protect the syslog ports by adding one to
one firewall rules. This can be performed at both the network and host layer.
Caching Capabilities
One other option is to add some caching capabilities on this
server by adding another destination of a local file. With enough disk space,
it could provide some time to collect logs from the syslog-ng server before
they rolled in the case of a network issue between the syslog-ng server and the
downstream host.
Conclusion
In this article we showed how we can set up an intermediary
syslog-ng server to split a single feed into multiple feeds. As a bonus, we
also showed how we could secure the syslog traffic by enabling TLS to wrap the
data with encryption. We hope this article helps get you up and running quicker
when you need to split and secure that next feed.
Thanks
Zack Link for the original write up using Red Hat Linux. Syslog-ng
team for the great software and documentation (https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.16/mutual-authentication-using-tls#TOPIC-956369).
