Lync Server: Windows Fabric Logs, Disk Space Usage and Circular Logging


If you're not interested in reading the entire post, you can skip to the solution.

I know there have been many articles written and available for this topic. The reason I decided to still write this is to provide more explanation to possible questions that I found myself asking too when I first encountered this situation.

To understand what Windows Fabric is, what is does and how it works you may refer to this link -
"Understanding how Windows Fabric Works (with regards to Lync)"

There are two (2) User Defined Collector sets by default when a Lync Front-End server is deployed.
  • FabricLeaseLayerTraces
  • FabricTraces
You can find these in Performance Monitor



And the default location of the files generated by these data collectors is in C:\ProgramData\Windows Fabric\Fabric\log\Traces folder of each Front-End server.


The screenshot above is from one of the FE Servers in my lab setup. As you can see there are multiple files with approximately 128MB size each. What happens is that once the log file reached a new log file is generated with an incremental version.

eg.
fabric_traces_130374156934210426_007625.etl
fabric_traces_130374156934210426_007626.etl

And there is no limit as to how many of these files are created and can ultimately use up all available free space.

So the questions:

Why the 128MB Maximum file size?

It can be any size really, it's just that 128MB is the default when the data collector was created. You can simply check this using the logman utility.


What triggers the data collector to start?

A Task is scheduled to run at system startup which calls the "StartTracing.cmd" batch file.




Where can I find StartTracing.cmd and how is Task Scheduler able to call it when there is no relative path indicated in the Task?

A relative path (eg. C:\Windows\etc\whatever.exe) is not necessary because the file StartTracing.cmd is located in a path that is already defined in the Environment Variables.



Think of notepad for example which you can run by just typing notepad.exe without having to specify the path - that is the same concept.

How do I stop Windows Fabric from generating multiple log trace files?

The short answer is to turn on Circular Logging.

How? According to this article - http://flinchbot.com/2014/02/28/the-hidden-logs-that-could-crash-your-lync-servers/ - you can turn on circular logging by running this command (from an elevated command prompt)
Logman update trace FabricLeaseLayerTraces -f bincirc --cnf
I did use that article as a guide to enable the Circular Logging for both collectors, but somehow logman still kept on generating new files. I'm not saying that it is wrong, only that some parameters were not included to achieve my intended outcome.

So why didn't it work? That is because despite circular logging being turned on, the versioning is still enabled and the overwrite switch was not specified. In order to really turn on Circular Logging and keep the number of files to 1, this is what I did:

Solution
  • Run these commands in an elevated command prompt:
Logman update trace FabricLeaseLayerTraces -f bincirc --v --cnf -ow
Logman update trace FabricTraces -f bincirc --v --cnf -ow
What do the switches mean?

-f bincrc = turn on circular logging
--v = turn off file versioning
--cnf = turn off creation of new files
-ow = turn on overwrite of existing log file
  • Temporarily stop the logging
logman stop fabricleaselayertraces
logman stop fabrictraces
  • (Optional) Delete the existing log files 
del "C:\ProgramData\Windows Fabric\Fabric\log\Traces\*.*"
  • Start logging
logman start fabricleaselayertraces
logman start fabrictraces
The Result?

Notice that the versioning has been removed.


The same log files will be overwritten every time.

Also, you may want to consider increasing the maximum file size to a much higher value (like 2GB or higher) since there will only one log file for each data collector.
logman update fabrictraces -max 2048
logman update fabricleaselayertraces -max 2048
References:

https://technet.microsoft.com/en-us/library/bb490956.aspx
http://masteringlync.com/2013/10/29/understanding-how-windows-fabric-works/
http://flinchbot.com/2014/02/28/the-hidden-logs-that-could-crash-your-lync-servers/
https://social.technet.microsoft.com/wiki/contents/articles/29408.out-of-disk-space-in-lync-server-2013.aspx



Share:

1 comment:

  1. Thanks for this. The path for deleting the logs on the servers I was on was "C:\ProgramData\Windows Fabric\log\Traces\*.*"

    ReplyDelete

Popular Posts

Powered by Blogger.