19 October 2014

Have you ever worked with an application that has it’s own log files? How do you get those logs into syslog on a remote system? Let’s walk through the process.

Note that even though the systemd-journal is included in RHEL / CentOS 7 the OS still uses syslog by default.

In this example we have two servers:

  • test1 is the client and has an IP address of
  • test2 is the server and has an IP address of

For simplicity’s sake let’s assume that the client and the server can talk to each other (eg: iptables / firewalld / routing / hardware firewall isn’t blocking the communications).

First, let’s tell our syslog server to listen for incoming connections. On the server test2 edit /etc/rsyslog.conf and change

# Provides TCP syslog reception
#$ModLoad imtcp
#$Input TCPServerRun 514


# Provides TCP syslog reception
$ModLoad imtcp
$Input TCPServerRun 514

and then bounce rsyslog to have it re-read the configuration file:

systemctl restart rsyslog.service

and then verify it’s listening it’s listening on port 514 TCP

netstat -lnp | grep -i rsyslog

Next, let’s move to the client, test1. Let’s create a little script that generates a fake log file. I used /root/log.sh


while true ; do
  echo 'exception in module blah blah' >> /opt/app/log/applog.log
  sleep 1
  echo 'WARNING: Helper process PID 1234 died!' >> /opt/app/log/applog.log
  sleep 1
  echo "User 'bob' logged in at 13:13" >> /opt/app/log/applog.log
  sleep 1

Now create the directory to hold our test logs:

mkdir -p /opt/app/log

Next we need to configure rsyslog on the client (test1) to monitor our log file. Edit /etc/rsyslog.conf and add the following lines:

## Enable file monitoring
$ModLoad imfile
$InputFilePollInterval 10

## This section monitors the log file /var/app/log/applog.log
$InputFileName /opt/app/log/applog.log
$InputFileTag MYAPP
$InputFileStateFile stat-MYAPP
$InputFileSeverity debug
$InputFileFacility local0
$InputFilePersistStateInterval 20000

$template MyApp-format,"%HOSTNAME%", %msg%\n"

if $programname == 'MYAPP'\
and $syslogfacility-text == 'local0'\
and $syslogseverity-text == 'debug'\
then @@;MyApp-format
& ~

Remember that is the IP of my server (test2). Why did I choose debug instead of info? Because by default all info messages are put into /var/log/messages (look at the *.info line in /etc/rsyslogd.conf) so we would end up with three copies (applog.log and syslog on both systems). Depending on your environment you might want to use a different facility and severity.

Next bounce rsyslog to have it re-read the configuration file

systemctl restart rsyslog.service

Now start our log generator script on test1 to have it start spitting out messages


Last, check syslog on test2 (our server) to make sure it’s getting the messages:

tail -f /var/log/messages

You should see our messages from /opt/app/log/applog.log on host1 in /var/log/messages on host2.

One last note: Why do the logs on the server show an IP address and not a FQDN? This question on ServerFault talks about it a bit.

There are tons of other options or ways to do this but hopefully this little primer will get your started. Thanks for reading!