Sunday, April 10, 2016

Using log4j2 filter plugin for a separate errors log

The log4j2 filter controls output to the log. There is a plenty of ready for use filters as described in filters manual.

A filter allows or denies messages according to its filtering criteria. For example, the ThresholdFilter masks messages, which log level is above or beyond the threshold log level.

A filter activation is done with the log4j2 configuration.

A filter has a particular scope: configuration, appender or logger. The scope is defined according to the filter location in the log4j2 configuration file

The only development effort needed for using the build-in log4j2 filter is to define appropriate log4j2 configuration. There is no code needed.

A custom filter should be done as a filter plugin.

Creating an errors log with the ThresholdFilter

The configuration defines an appender filter, which drops all messages except errors:
 <RollingFile name="ErrorsLog" fileName="${errLogDir}/${errLogFile}.log"  
      filePattern="${errLogDir}/${errLogFile}-%d{dd-MM-yyyy}.log">  
      <ThresholdFilter level="error"/>  
      <PatternLayout>   
           <pattern>  
                %d{EEE, dd MMM yyyy HH':'mm':'ss}|%m%n  
           </pattern>  
      </PatternLayout>  
 </RollingFile>  

Filtering messages with RegexFilter

The RegEx filter is defined with the log4j2 configuration. It allows messages with word "CRITICAL" and drops the rest of messages:
 <Appenders>  
    <Console name="SysOut">  
       <RegexFilter regex=".*\bCRITICAL\b.*" useRawMsg="true" onMatch="ACCEPT" onMismatch="DENY"/>  
       <PatternLayout>  
          <pattern>  
             %p %config %msg  
          </pattern>  
       </PatternLayout>  
    </Console>  
 </Appenders>  
RegexFilter may be created explicitly with the code:
 public static RegexFilter createRegExpFilter(String pattern, Result onMatch, Result onMismatch) throws IllegalArgumentException, IllegalAccessException {  
       return RegexFilter.createFilter(pattern, null, false, onMatch, onMismatch);  
}  
Instead of null the second argument may contain regex operations as a String[].

Creating of filter plugin for errors messages


Error filters in the examples above are not flexible enough: the ThresholdFilter detects only log level, the RegexFilter looks for the word "CRITICAL", which is not strictly unique.

The filter plugin ErrorPatternsFilter implements the full error log functionality:
- It recognizes the particular error patterns and produces the separate log file for errors, matching the patterns.
- It prevents appearance of the matching errors in the usual log.
- It supports configurable error patterns: the filter plugin may be instantiated for different error patterns.

The implementation includes:
  1. Class ErrorPatternsFilter extends the AbstractFilter.
    It is annotated with the @Plugin annotation. The annotation attribute "name" is the name to be mentioned in the log4j2 configuration.
    The matchingValues variable holds the values to be matched by the filter.
    The filter drops or allows the matching message according to the dropOnMatch flag.
     @Plugin(name = "ErrorPatternsFilter",
          category = Node.CATEGORY,   
          elementType = Filter.ELEMENT_TYPE,
          printObject = true)  
     public class ErrorPatternsFilter extends AbstractFilter {  
       private Set<String> matchingValues;  
       private ErrorPatternsFilter(final Set<String> values, final boolean dropOnMatch) {  
            super(dropOnMatch ? Result.DENY : Result.NEUTRAL,  
                 dropOnMatch ? Result.NEUTRAL: Result.DENY);  
            this.matchingValues = values;  
       }  
     ...  
       private Result filter(final String msg) {  
          if (msg == null) {  
               return onMismatch;  
          }  
          boolean matchFound = false;  
          for (String value : matchingValues) {  
               if (msg.indexOf(value) != -1) {  
                    matchFound = true;  
                    break;  
               }  
          }  
          return matchFound ? onMatch : onMismatch;  
       }  
     ...  
     }  
    
  2. Interface ErrosPatterns defines an API for the matching values and decouples the filter implementation from the concrete matching values:
     public interface ErrorsPatterns {  
          String[] values();  
     }  
    

  3. The matching values are initialized upon filter creation with the external class that implements the ErrosPatterns interface:
     private static Set<String> fillMatchingValues(final String errorsPatternsClassName) {  
          Set<String> matchingValues = new HashSet<String>();  
          try {  
               Class<?> classTemp = Class.forName(errorsPatternsClassName);  
               ErrorsPatterns errorsPatterns = (ErrorsPatterns)classTemp.newInstance();  
               for (String pattern : errorsPatterns.values()) {  
                    matchingValues.add(pattern);  
               }  
          }   
          catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) {  
               LOGGER.error("Failed to init matching values for a filter:", e);  
          }  
          catch(ClassCastException e) {  
               LOGGER.error("Invalid errors patterns class was provided:", e);  
          }  
          return matchingValues;  
     }  
    
  4. A filter instance may be created only with the plugin factory method. The factory method createFilter defines two plugin attributes:
    - errorsPattern contains the name of the class, which provides the matching values
    - dropOnMatch is a flag, which defines the filter behavior upon matching
     @PluginFactory  
     public static ErrorPatternsFilter createFilter(  
       @PluginAttribute("errorsPatterns") final String errorsPatternsClassName,  
       @PluginAttribute("dropOnMatch") final Boolean dropOnMatch) {  
          final Set<String> matchingValues = fillMatchingValues(errorsPatternsClassName);  
          return new ErrorPatternsFilter(matchingValues, dropOnMatch);  
     }  
    
  5. The log4j2 configuration should define:
    - The filter plugin location in the project. For example:
     <Configuration packages="com.alasch1.logging.plugins">  
    

    - The filter configuration: the filter class name and filter attributes values. Filter attributes are mentioned like key=value, where key is same to declared in the plugin factory method. For example:
     <Filters>  
          <ErrorPatternsFilter errorsPatterns="com.alasch1.logging.mocks.ErrorsPatterns4Tests" 
                               dropOnMatch="true"/>  
     </Filters>
    

The example of errors pattern implementation.

The ErrorsPatterns4Tests implements the ErrorsPatterns:
 package com.alasch1.logging.mocks;  
 import com.alasch1.logging.api.ErrorsPatterns;  
 public class ErrorsPatterns4Tests implements ErrorsPatterns {  
      public static String ERROR_PATTERN_1 = "AAA_BBB_CCC";  
      public static String ERROR_PATTERN_2 = "1234567_OOOOO";  
      public static String ERROR_PATTERN_3 = "Harry Potter";  
      public static String[] TEST_VALUES = {ERROR_PATTERN_1, ERROR_PATTERN_2, ERROR_PATTERN_3};  
      @Override  
      public String[] values() {            
           return TEST_VALUES;  
      }  
 }  
The log4j configuration defines two appenders:
- The AppLog is a normal application log appender. The error patterns filter is applied with an attribute dropOnMatch=true and prevent the error messages to be printed into it.
- The ErrorsLog is an error log appender. The error patterns filter is applied with an attribute dropOnMatch=false and only matching messages are printed into it.
- The appenders use the different format for messages.
 <RollingFile name="AppLog" fileName="${logDir}/${logFile}.log"  
      filePattern="${logDir}/${logFile}-%d{dd-MM-yyyy}-%i.log">  
      <Filters>  
           <ErrorPatternsFilter errorsPatterns="com.alasch1.logging.mocks.ErrorsPatterns4Tests" dropOnMatch="true"/>  
      </Filters>  
      <PatternLayout>   
           <header>  
                ${java:runtime}${sys:line.separator}${java:vm}${sys:line.separator}${java:os}${sys:line.separator}${java:hw}  
                ${sys:line.separator}$${config:all}${sys:line.separator}  
           </header>  
           <footer>  
                End of file  
           </footer>  
           <pattern>  
                %d %p [%X{${tag1}}|%X{${tag2}}|%X{${tag3}}|%X{${tag4}}|%X{${tag5}}|%X{${tag6}}] (%c{1.}.%M:%L)- %m%n  
           </pattern>  
      </PatternLayout>  
      <Policies>  
           <TimeBasedTriggeringPolicy/>  
           <SizeBasedTriggeringPolicy size="1 MB"/>  
      </Policies>  
      <DefaultRolloverStrategy max="21"/>  
 </RollingFile>  
 <RollingFile name="ErrorsLog" fileName="${errLogDir}/${errLogFile}.log"  
      filePattern="${errLogDir}/${errLogFile}-%d{dd-MM-yyyy}.log">  
      <Filters>  
           <ErrorPatternsFilter errorsPatterns="com.alasch1.logging.mocks.ErrorsPatterns4Tests" dropOnMatch="false"/>  
           <ThresholdFilter level="error"/>  
      </Filters>  
      <PatternLayout>   
           <pattern>  
                %d{EEE, dd MMM yyyy HH':'mm':'ss } %m%n  
           </pattern>  
      </PatternLayout>  
      <Policies>  
           <TimeBasedTriggeringPolicy/>  
      </Policies>  
      <DefaultRolloverStrategy max="30"/>  
 </RollingFile>  
All the sources on Git.

No comments :

About the author

My Photo
I trust only simple code and believe that code should be handsome. This is not a matter of technology, but professional approach, consolidated after years of software development. I enjoy to cause things working and feel very happy, when I manage to solve a problem.
Back to Top