Cleanup strategy missing WebDAV, Search and Cragling

What about the cleanup strategy for WebDAV.log, Search.log and Crawling.log, Sitecore?


Whenever Sitecore version 7.X restarts, it creates four new log files. The naming and location of these log files are managed in the <log4net> setting in the web.config file. The first log file Sitecore creates is the system log file. This log file is named log.[yyyyMMdd].txt and is managed at the "LogFileAppender". The second log file is the WebDAV log file named WebDAV.log.[yyyyMMdd].txt handled by the "WebDAVLogFileAppender". The third handles the search indexing operations named Search.log.[yyyyMMdd].txt and fourth handles the logging for search query operations named Crawling.log.[yyyyMMdd].txt and are managed at the "SearchLogFileAppender" and "CrawlingLogFileAppender" respectively.

If Sitecore restarts more than once a day, the time of creation (HHmmss) will be added to the all subsequent log files for that specific day E.g. log.[yyyyMMdd].[HHmmss].txt.

The "Appender" section contains:

    
<appender name="LogFileAppender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
    <file value="$(dataFolder)/logs/log.{date}.txt">
        <appendtofile value="true">
            <layout type="log4net.Layout.PatternLayout">
                <conversionpattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
            </layout>
        </appendtofile>
    </file>
</appender>
<appender name="WebDAVLogFileAppender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
    <file value="$(dataFolder)/logs/WebDAV.log.{date}.txt">
        <appendtofile value="true">
            <layout type="log4net.Layout.PatternLayout">
                <conversionpattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
            </layout>
        </appendtofile>
    </file>
</appender>
<appender name="SearchLogFileAppender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
    <file value="$(dataFolder)/logs/Search.log.{date}.txt">
        <appendtofile value="true">
            <layout type="log4net.Layout.PatternLayout">
                <conversionpattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
            </layout>
        </appendtofile>
    </file>
</appender>
<appender name="CrawlingLogFileAppender" type="log4net.Appender.SitecoreLogFileAppender, Sitecore.Logging">
    <file value="$(dataFolder)/logs/Crawling.log.{date}.txt">
        <appendtofile value="true">
            <layout type="log4net.Layout.PatternLayout">
                <conversionpattern value="%4t %d{ABSOLUTE} %-5p %m%n" />
            </layout>
        </appendtofile>
    </file>
</appender>

If you compare the "appender" settings listed above with the "Sitecore.Tasks.CleanupAgent" placed at <sitecore> <scheduling> <agent> setting you will notice that the CleanupAgent does not contain any remove settings regarding the log files of WebDAV, Search or Crawling. Put in another way, by default Sitecore does not clean up this three log files, which results in a solution with an unmanageable number log files (If you take this into a Sitecore version 6.X, you get the same regarding the WebDAV log files).

It is easy to set up Sitecore to clean up the three additional log files. You could simply prefix "log.*.txt" with "*." at:

            
<remove folder="$(dataFolder)/logs" pattern="log.*.txt" maxcount="50" minage="30.00:00:00" />

Ending up with this:

            
<remove folder="$(dataFolder)/logs" pattern="*.log.*.txt" maxCount="50" minAge="30.00:00:00" />

In this case, the three additional log files will follow the same life cycle as the system log file. If you want specific life cycles for each of the log files you could add specific "remove" settings for each of the log files ending up with something like this:

<files hint="raw:AddCommand">
    <remove folder="$(dataFolder)/logs" pattern="log.*.txt" maxcount="50" minage="30.00:00:00" />
    <remove folder="$(dataFolder)/logs" pattern="WebDAV.log.*.txt" maxcount="20" minage="7.00:00:00" />
    <remove folder="$(dataFolder)/logs" pattern="Search.log.*.txt" maxcount="20" minage="7.00:00:00" />
    <remove folder="$(dataFolder)/logs" pattern="Crawling.log.*.txt" maxcount="20" minage="7.00:00:00" />
… 
</files>

In this case, Sitecore will store the system log file at least for 30 days, if the number of system log files does not exceed the number of 50, while Sitecore will only store the log files of WebDAV, Search and Crawling for seven days.

If you are not using WebDAV in your Sitecore solution, you can easily disable it. This is how you disable WebDAV.