Sometimes you may want to route your log messages to Syslog in UNIX-based environment like Linux. Since logger allows multiple appenders, you can print the same log messages to a log file, console, and route it to Syslog at the same time. In order to send log messages to Syslog using log4j2, you need to make some changes in your log4j2.xml file. This change is to include the Syslog appender and configure that, once you do that you also need to enable TCP or UDP reception by editing rsyslog.conf file. For faster transmission you can choose UDP protocol and configure a port number on which your Syslog is listening, In our example, Syslog is listening on port 618. Let's see the complete step-by-step guide to configure Syslog logging in log4j logger.
Syslog appender can only be used with two layouts RFC5424 or BSD, the format attribute is used to specify that. If set to "RFC5424" the data will be formatted in accordance with RFC 5424. Otherwise, it will be formatted as a BSD Syslog record. Note that although BSD Syslog records are required to be 1024 bytes or shorter the SyslogLayout does not truncate them.
The RFC5424Layout also does not truncate records since the receiver must accept records of up to 2048 bytes and may accept records that are longer. the protocol can be either TCP or UDP, accordingly, you need to edit /etc/rsyslog.conf file to enable TCP or UDP reception.
2. Depending upon, which protocol you are using to publish log messages to Syslog, enable them into file /etc/rsyslog.conf
Since we are using UDP to publish log messages to Syslog, we have enabled UDP Syslog reception, by the way, you may need to use root login to edit this file. You can use su root to switch to the root login. Pay attention to port value 514 as well, you need to provide the same port number to your syslog appender in log4j2.xml using the port attribute, as shown above, port="514" protocol="UDP".
$ /etc/init.d/rsyslog restart
Now, if you see this you will find that output is really simple, it's only printing log messages and not useful metadata like class, thread or even logging level. Thankfully you can configure that using LoggerFields nested element for RFC5424Layout. Just change your Syslog appender as following :
This will print formatted input, where values are taken from the MDC data structure, which is a HashMap.
You can customize the format depending upon your need like if you only need logging level then you can just put %p in value.
Since windows environment doesn't have a Syslog facility, you can use a networking tool that can listen to UDP messages on a port, socat is one such tool, you can run socat as
$ socat stdio udp-recv:514
to receive UDP messages sent to port 514. You can further see these free Linux courses to learn more about essential Linux commands.
In order to confirm that whether your log4j setup is correct or not, I recommend listening to UDP messages on port 514, since syslogger also listens on the same port, please stop it before starting your program, maybe a Perl or python script or even a Java program to listen to UDP traffic, failing to do so may throw error "address already in use".
If you see structured data correctly then your log4j2 syslog appender is correctly set up and publishing data, and it's rsyslog which is not supporting full RFC5424 format, especially structured data. On the other hand, if you don't see structured data and only see space e.g. (-) then it's time to look at your Syslog configuration in log4j2.xml file. You must enable includeMDC field as true.
Other Java Logging Tutorials and Best Practices you may like
How to send logger messages to Syslog using Log4j2 SysLogAppender in Linux
Here is the step by step instructor to configure Log4j for your Java application to send login messages to SysLog in the Linux environment:1. Configure SysLogAppender in log4j2.xml file
<Syslog name="Syslog" format="RFC5424" host="localhost" port="514"
protocol="UDP" appName="yourApp" newLine="true" />
Syslog appender can only be used with two layouts RFC5424 or BSD, the format attribute is used to specify that. If set to "RFC5424" the data will be formatted in accordance with RFC 5424. Otherwise, it will be formatted as a BSD Syslog record. Note that although BSD Syslog records are required to be 1024 bytes or shorter the SyslogLayout does not truncate them.
The RFC5424Layout also does not truncate records since the receiver must accept records of up to 2048 bytes and may accept records that are longer. the protocol can be either TCP or UDP, accordingly, you need to edit /etc/rsyslog.conf file to enable TCP or UDP reception.
2. Depending upon, which protocol you are using to publish log messages to Syslog, enable them into file /etc/rsyslog.conf
vi /etc/rsyslog.conf # Provides UDP syslog reception $ModLoad imudp.so $UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp.so #$InputTCPServerRun 514
Since we are using UDP to publish log messages to Syslog, we have enabled UDP Syslog reception, by the way, you may need to use root login to edit this file. You can use su root to switch to the root login. Pay attention to port value 514 as well, you need to provide the same port number to your syslog appender in log4j2.xml using the port attribute, as shown above, port="514" protocol="UDP".
3. Restart Syslog in Linux
$ /etc/init.d/rsyslog restart
4. Tail Syslog
Now, it's time to see the output by tailing the Syslog file, you will most likely seetail -f /var/log/messages
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Now, if you see this you will find that output is really simple, it's only printing log messages and not useful metadata like class, thread or even logging level. Thankfully you can configure that using LoggerFields nested element for RFC5424Layout. Just change your Syslog appender as following :
<Syslog name="Syslog" format="RFC5424" host="localhost" port="514" protocol="UDP" appName="This is Warning messages, printed at WARN level" newLine="true" /> <LoggerFields> <KeyValuePair key="" value="%d{yyyy-MM-dd HH:mm:ss,SSS} [%t] %-4p %c " /> </LoggerFields> </Syslog>
This will print formatted input, where values are taken from the MDC data structure, which is a HashMap.
Output
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp
- [mdc@27030 ="2019-11-18 11:46:43,011 [pool-1 thread-1\]
WARN com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN
com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN
com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN
com.startup.Hello"] This is Warning messages, printed at WARN level
Since windows environment doesn't have a Syslog facility, you can use a networking tool that can listen to UDP messages on a port, socat is one such tool, you can run socat as
$ socat stdio udp-recv:514
to receive UDP messages sent to port 514. You can further see these free Linux courses to learn more about essential Linux commands.
Issue: Not seeing Structured data in Syslog
Even after correct setup and using LoggerFields, if you don't see structured data required to show metadata related to logging statements like log level, category, thread, and mapped diagnostic context data, it's time to check if your rsyslog supports structured data or not. Apparently, older versions of rsyslog, below 5.6.6 are not supporting structured data, so if you are running on the older version, it's time to update rsyslog binaries as well.In order to confirm that whether your log4j setup is correct or not, I recommend listening to UDP messages on port 514, since syslogger also listens on the same port, please stop it before starting your program, maybe a Perl or python script or even a Java program to listen to UDP traffic, failing to do so may throw error "address already in use".
If you see structured data correctly then your log4j2 syslog appender is correctly set up and publishing data, and it's rsyslog which is not supporting full RFC5424 format, especially structured data. On the other hand, if you don't see structured data and only see space e.g. (-) then it's time to look at your Syslog configuration in log4j2.xml file. You must enable includeMDC field as true.
Other Java Logging Tutorials and Best Practices you may like
- 5 Java Performance tuning books for experienced Programmers (list)
- Why use Log4j logging vs System.out.println in Java? (answer)
- Why use SLF4j over log4j for logging in Java? (answer)
- How to enable SSL debugging log in Java Virtual Machine? (tips)
- How to configure Log4j without XML or Properties file in Java? (tutorial)
- Difference between Functional and Non-Functional Requirement (article)
- 10 Essential JVM Options for Real-world Java Applications (tips)
- How to use MDC or Mapped Diagnostic Context in Log4j? (tutorial)
- 10 Tips for logging in Java application (tips)
- How to fix Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory in Java (solution)
No comments:
Post a Comment
Feel free to comment, ask questions if you have any doubt.