A closer look at Smart File Defragmentation

In my previous blog post I mentioned the "Freespace Consolidation" and "Smart File Defragmentation" components in Defrag.NSF V2. So, how smart is Defrag.NSF V2 and what will the end result differences be when comparing the kind of intelligent strategic file placement Defrag.NSF V2 uses, compared to simply using a freely available tool like contig.exe?

We demonstrate this in the pics below, the difference is in fact quite remarkable in the final result, between each of these runs the server was restored back to the same fragmented starting point each time.

Remember when viewing these pictures that:

- Red bars are bad.

- Small Blue bars are good.

- Large Blue bars are great

- Large White bars are what we are after along with Large Blue bars.

Comment - So in the beginning the volume is a mess.

Figure 2 - After contig.exe has completed its run.

Comment: Files are nicely defragged, but freespace is a mess. New new large files will fragment.

Figure 3 - After Defrag.NSF V2 completed in "Fast" mode.

Comment: File are nicely defragged and freespace is very usable.

Figure 4 - After Defrag.NSF V2 completed "Freespace Consolidation" mode

Comment: Perfect ! Please note small sliver of white space is actually the MFT !

Defrag.NSF V2 -Why freespace defragmentation is also important

One of Defrag.NSF's primary functions is file level defragmentation. This is because a Domino database typically represents a single file on disk.

One of the historical issues with file level defragmentation products is that they need a large enough freespace block on disk to move the file into. This can get a bit tricky when people have large (think 25GB+) mail files.

Defrag.NSF version 1, uses a number of different techniques to perform this relocation, including the ability to move to multiple freespace holes if a suitably large enough one did not exist. There is however a chance that even though there is plenty of freespace on a volume, it just may not be usable.

Let look at an example. In figure 1, there is plenty of freespace, and it consists of two very large contiguous sections. This is a great scenario.

Figure 1.

In figure 2, we have the same amount of freespace, but from a classic file defragmentation point of view it is basically unusable for large database defrags. The freespace itself is fragmented.

Figure 2.

So to help out with this exact situation, Defrag.NSF V2 is going to include two new features:

1) Freespace Consolidation - the ability to pull back all the freespace into a single chunk so it can be used, in effect, defragmenting the freespace

2) Smart File Defragmentation - the ability to intelligently defrag a file by strategic relocation of other nearby files, and allowing optimum usage of the freespace adjacent to the original file's current location.

Over the next two weeks I'll step you through how these features work as we prepare to release Defrag.NSF V2.

Defragging NTFS on a SAN DOES make a difference

I noticed in one of my previous posts, a comment on the fact that because SANs typically fragment their data physically over multiple disks, then they are "fragmented by design" and hence do not benefit from being defragmented.

Unfortunately the author of that post does not understand the difference between defragging the SAN and defragging the NTFS volume on the SAN.

In simple terms, if NTFS is fragmented, Windows has to issue more I/O requests (spilt I/O etc). These I/O requests added up very quickly and slow the overall performance of the OS and the SAN. In this scenario, it is Windows perception of the disk that matters. It spends a significant amount of time blocking and waiting on I/Os that just aren't required.

At the end of the day, the evidence also points to the fact that defragging makes a difference. Time and time again the results are the same. The clock does not lie.

If your SAN administrators tell you otherwise, then ask them to do a test... but be careful some people don't like to be proven wrong.

If it's your Domino server... give Defrag.NSF a try.

Lotus Knows suggestion - the Swiss Army Knife approach

Do you think it would work ? I think it's a winner !!!

File fragmentation of 98% appears to be common

I've just done some sampling on a number of Domino 8.5.1 servers that have not be defragmented since they were build (2+ years) and it appear that 98% file fragmentation is rather common.

Further proof that when left to its own, NTFS and Domino can make a rather nasty mess.

So if you have never done a Defrag, startup the Windows defragmentation tool, point it to your Domino data volume and run an analyse. Let me know what you find.

If it is a nasty mess, give Defrag.NSF a try.

Mail Database fragmentation over time

We've been busy working on version 2.0 of Defrag and some of its awesome new features (more on that soon) and during this testing process we wanted to find out just how fast fragmentation occurred on a perfectly clean Domino Server.

We used IBM's testing and simulation tools to represent 200 users sending and receiving mail over a 7 hour period. We stopped ever hour and measured the results, which confirmed things can get rather bad, very quickly. Please see the graph below.

If you'd like to know more about Defragging Domino databases, please see our website and register for a FREE trial.

Updating bookmarks with policies appears to be broken

We've been revisiting updating bookmarks via Desktop policies and it appears that in 8.5.1 it is (still?) broken.

When the policy is initially applied some users get the bookmark, some do not. We have tried the debugging steps outlined in IBM's Troubleshooting policies and settings document. These steps sometimes work but more often than not don't.

Interestingly in the 8.5.1 contacts database an number of the agents that the technote tells you to run no longer exist.

Another interesting feature is that if a user gets a bookmark icon added, and then they deleted it, there appears to be no way to add it back via a policy.

Has anyone actually got bookmarks and policies working together nicely ? If you could point me in the right direction I'd greatly appreciate it.