Using Loggly with Java/Tomcat on shared host

You may sometimes need an easy access to your Tomcat logs outside of the hosting server. Loggly is one of the solutions but setting it up on a shared server is not straightforward.

Loggly installation guide assumes you have superuser privileges (VPS/dedicated server) so that Tomcat is installed system-wide and rsyslog.conf can be easily modified. If it is not the case and you are for example using private JVM on shared host read this tutorial to have your Tomcat working with Loggly in a few rather simple (we hope ;) steps.

Basically you will need your private rsyslogd service running on one of your custom ports. Check cPanel – Java Control Panel – Ports for an unused port number or request support to assign you one. The testing environment for the below example was Centos 6 64-bit. User home directory was /home/tomcat – change it in all occurences below to your value.

1) Preparing for running rsyslog/Loggly in your home directory

- Register for Loggly account – it may be trial.
- Make sure compiler is available on your account. If not the ask support to enable it.
- Make sure outgoing TCP port 514 is open in firewall for the user. Consult with support.

2) Building rsyslogd

Default rsyslog version in Centos 6 is 5.8.10 so let’s use neweset 5.8.x for highest compatibility.

tar xzf rsyslog-5.8.13.tar.gz
cd rsyslog-5.8.13

Imfile module will be needed to trace Tomcat log file. It uses inotify by default. As imfile is off by default we need to enable it in configure line.

./configure --prefix=$HOME --exec-prefix=$HOME --enable-imfile
make install
mkdir -p ~/var/{spool/rsyslog,run}
echo 'export PATH=~/sbin:$PATH' >> ~/.bashrc && . ~/.bashrc
which rsyslogd

Create ~/etc/rsyslog.conf – replace TOKEN with your Loggly token and $WorkDirectory with your own path.

echo '#LOGGLY
# uncomment the below 3 lines if using Log4j + UDP
#$UDPServerRun your_custom_port_here
$WorkDirectory /home/tomcat/var/spool/rsyslog
$template LogglyFormat,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [TOKEN@41058 tag=\"MyRsyslogd\"] %msg%\n"
' > ~/etc/rsyslog.conf

Do not forget to update paths in all $InputFileName lines and TOKEN.
Start your own rsyslog with:

rsyslogd -c5 -4 -i ~/var/run/ -f ~/etc/rsyslog.conf

There should be no error diplsayed and the process will go to the background. Check for first messages in loggly.

3) Adjusting Tomcat logging for Loggly

In conf/ add the 4 rotatable=false lines and remove periods from names like in the following patch:

echo '---    2014-11-10 06:13:32.978341583 -0600
+++    2014-11-10 06:18:09.670432130 -0600
@@ -24,19 +24,23 @@ = FINE = ${catalina.base}/logs = catalina. = catalina = false = FINE = ${catalina.base}/logs = localhost. = localhost = false = FINE = ${catalina.base}/logs = manager. = manager = false = FINE = ${catalina.base}/logs = host-manager. = host-manager = false
 java.util.logging.ConsoleHandler.level = FINE
 java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter' >
patch -p0 < $CATALINA_HOME/conf/

This way you will have catalina.log, catalina.out, host-manager.log, localhost.log and manager.log in your logs directory. Note the names no more contain variable (date) part (after Tomcat restart).

Enable montoring of the files in rsyslog.conf by appending the follwing lines to ~/etc/rsyslog.conf.

echo '

$ModLoad imfile
$InputFilePollInterval 10

#Add a tag for tomcat events
$template LogglyFormatTomcat,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [TOKEN@41058 tag=\"tomcat\"] %msg%\n"

# catalina.log
$InputFileName /home/tomcat/appservers/apache-tomcat-7.0.55/logs/catalina.log
$InputFileTag catalina-log
$InputFileStateFile stat-catalina-log
$InputFileSeverity info
$InputFilePersistStateInterval 20000
if $programname == 'catalina-log' then;LogglyFormatTomcat
if $programname == 'catalina-log' then ~

# catalina.out
$InputFileName /home/tomcat/appservers/apache-tomcat-7.0.55/logs/catalina.out
$InputFileTag catalina-out
$InputFileStateFile stat-catalina-out
$InputFileSeverity info
$InputFilePersistStateInterval 20000
if $programname == 'catalina-out' then;LogglyFormatTomcat
if $programname == 'catalina-out' then ~

# host-manager.log
$InputFileName /home/tomcat/appservers/apache-tomcat-7.0.55/logs/host-manager.log
$InputFileTag host-manager
$InputFileStateFile stat-host-manager
$InputFileSeverity info
$InputFilePersistStateInterval 20000
if $programname == 'host-manager' then;LogglyFormatTomcat
if $programname == 'host-manager' then ~

# localhost.log
$InputFileName /home/tomcat/appservers/apache-tomcat-7.0.55/logs/localhost.log
$InputFileTag localhost-log
$InputFileStateFile stat-localhost-log
$InputFileSeverity info
$InputFilePersistStateInterval 20000
if $programname == 'localhost-log' then;LogglyFormatTomcat
if $programname == 'localhost-log' then ~

# manager.log
$InputFileName /home/tomcat/appservers/apache-tomcat-7.0.55/logs/manager.log
$InputFileTag manager
$InputFileStateFile stat-manager
$InputFileSeverity info
$InputFilePersistStateInterval 20000
if $programname == 'manager' then;LogglyFormatTomcat
if $programname == 'manager' then ~
' >> ~/etc/rsyslog.conf<<EOF

Do not forget to update paths in all $InputFileName lines  and TOKEN.
Finally restart your rsyslogd and Tomcat:

kill -9 `pgrep -u $USER rsyslogd`
rsyslogd -c5 -4 -i ~/var/run/ -f ~/etc/rsyslog.conf

You are done! See the screenshot below. Messages from Tomcat log should now start coming to Loggly.


Another option is logging from Log4j to rsyslogd with UDP and we describe it below.

As for keeping rsyslog up it can either be started/stopped along with Tomcat or we can setup monit task that it will take care of it to be always up.

4) Switching to UDP and Log4j

To switch to Log4j we will comment out (or remove) all lines below ‘#TOMCAT’ in ~/etc/rsyslog.conf and uncomment the 3 lines at the top of the file:

$UDPServerRun your_custom_port_here

To restart rsyslogd:

kill -9 `pgrep -u $USER rsyslogd`
rsyslogd -c5 -4 -i ~/var/run/ -f ~/etc/rsyslog.conf

or to only reread config fie (if rsyslogd is running):

kill -HUP `pgrep -u $USER rsyslogd`

Then copy in Log4j jar (e.g. log4j-1.2.17.jar) into $CATALINA_HOME/lib unless you already have it there on inside WEB-INF/lb of a specific application you want to log for.
As we are making the change for whole Tomcat we will also need tomcat-juli-adapters.jar and tomcat-juli.jar from Tomcat ‘extras‘. Ensure you download from correct ‘extras‘ matching your Tomcat version.

rm -f tomcat-juli.jar

Create or modify – we will use fresh Tomcat-wide version of the file placed in lib directory.

echo 'log4j.rootLogger=INFO, SYSLOG
log4j.appender.SYSLOG.layout.ConversionPattern=java %d{ISO8601} %p %t %c{1}.%M - %m%n

The above example only ensures logging to Loggly. Read more here about fully replacing Tomcat logging mechanism with Log4j at if you also want to have logs preserved in Tomcat’s log directory and displayed on console (standard output).

Restart Tomcat and you should see new logs in Loggly.


Instead of using ‘-i’ switch to indicated pidfile path you could update sources to have pid file in your home directory with this patch.

cat > syslogd.c.patch<<EOF 
--- tools/syslogd.c.orig    2014-11-10 05:53:45.304901299 -0600
+++ tools/syslogd.c    2014-11-10 04:31:58.790985752 -0600
@@ -195,6 +195,9 @@
 #    endif
+#define _PATH_VARRUN "/home/tomcat/var/run/"
 #ifndef _PATH_TTY
 #    define _PATH_TTY    "/dev/tty"
patch -p0 < syslogd.c.patch

To test UDP logging you may use newer version of ‘logger’ utility that supports -P (port) switch or use a simple Perl script:

#!/usr/bin/perl -w
use strict;
use Log::Syslog::Fast ':all';
my $logger = Log::Syslog::Fast->new(LOG_UDP, "", your_custom_port_here, LOG_LOCAL0, LOG_INFO, "myserver", "logger2");
$logger->send("log message", time);

This entry was posted in Java, Tomcat. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

four × = twenty

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>