Roberto Gallea Blog

Shibboleth IDP: Saving audit log to database

Shibboleth IDP has a flexible logging subsystem leveraging logback. Out of the box it writes several log files to filesystem, namely

  • idp-audit.log - detailed record of every request and response handled by the IdP to allow tracing of user activity
  • idp-process.log - information about idp activity
  • idp-warn.log - information about errors and warning

Official shibboleth idp page https://wiki.shibboleth.net/confluence/display/IDP30/LoggingConfiguration provides useful information for configuring logging. However I once needed a feature that was undocumented on that page. In order to process accesses data and provide access statistics, I needed to save audit logging to database. This is documented by logback but it is not explained in detail.

 

I won't go in detail into logback configuration, rather I will get to the point and illstrate the step required to save data into db.

 

The following steps are required:

  1. Create logback tables into DB
  2. Adding a DB appender to conf/logback.xml
  3. Edit the data format into conf/audit.xml

 

Create logback tables into DB

In order to use database logging you need to create specific tables required by logback. Fortunately, logback team provides script for the major DBMS. The SQL scripts are available at https://github.com/qos-ch/logback/tree/master/logback-classic/src/main/resources/ch/qos/logback/classic/db/script. The tables contain the following columns:

  • timestmp: timestamp of the event
  • formatted_message: the content of the event, this is the main informative column
  • logger_name: module that generated the event
  • level_string: event level (info, warn, etc)
  • thread_name: name of the thread generating the event
  • reference_flag: a small integer indicating if exceptions are present in the event
  • arg0, arg1, arg2, arg3: additional extra column, not used by shibboleth, afaik

 

Adding ad DB appender to conf/logback.xml

To enable DB logging you have to add a DB appender into file conf/logback.xml.

 

<!-- START DB APPENDER -->
<appender class="ch.qos.logback.classic.db.DBAppender" name="IDP_DB_APPENDER">
    <connectionsource class="ch.qos.logback.core.db.DataSourceConnectionSource">
      <datasource class="com.mchange.v2.c3p0.ComboPooledDataSource">
        <driverclass>DRIVER.CLASS.NAME</driverclass>
        <jdbcurl>JDBC_CONNECTION_URL</jdbcurl>
        <user>DB_USER</user>
        <password>DB_PASSWORD</password>
      </datasource>
    </connectionsource>
  </appender>
<!-- END DB APPENDER -->

Then add it to active loggers

<logger level="ALL" name="Shibboleth-Audit">
    <appender-ref ref="${idp.audit.appender:-IDP_AUDIT}"/>
    <appender-ref ref="IDP_DB_APPENDER"/>
</logger>

Edit the data format into conf/audit.xml

By default, Shibboleth IDP saves audit data in a pipe-delimited format. Such format is defined in the file conf/audit.xml.

<util:map id="shibboleth.AuditFormattingMap">
    <entry key="Shibboleth-Audit" value="%T|%b|%I|%SP|%P|%IDP|%bb|%III|%u|%ac|%attr|%n|%i|"></entry></util:map>

Each escaped codes define a field: the meaning of these codes are explained at https://wiki.shibboleth.net/confluence/display/IDP30/AuditLoggingConfiguration.

You can define your own format. I needed to store a JSON document as follows:

<util:map id="shibboleth.AuditFormattingMap">
    <entry key="Shibboleth-Audit" value="{&quot;timestamp&quot;:&quot;%T&quot;,&quot;inbound_binding&quot;:&quot;%b&quot;,&quot;inbound_message_id&quot;:&quot;%I&quot;,&quot;service_provider_name&quot;:&quot;%SP&quot;,&quot;profile_id&quot;:&quot;%P&quot;,&quot;identity_provider_name&quot;:&quot;%IDP&quot;,&quot;outbound_binding&quot;:&quot;%bb&quot;,&quot;outbound_message_id&quot;:&quot;%III&quot;,&quot;username&quot;:&quot;%u&quot;,&quot;authentication_context&quot;:&quot;%ac&quot;,&quot;attributes&quot;:&quot;%attr&quot;,&quot;nameid_value&quot;:&quot;%n&quot;,&quot;assertion_id&quot;:&quot;%i&quot;}"></entry>
</util:map>

That's all. After that you need to restart the logging service by visiting https://IDP_ADDRESS/idp/profile/admin/reload-service?id=shibboleth.LoggingService or by restarting the application container.

 

Conclusion

This article described how to enable database audit logging for Shibboleth IDP by editing the configuration of the logback subsystem. If you have comments or corrections, please leave your comments.

Categories

Tags