Adam Osborne's Blog

How to install Defrag.NSF for Notes

2011-12-07T14:22:00.001+11:00

The following steps show the Defrag.NSF for Lotus Notes install process.

Launch the MSI and follow the bouncy ball.

That's it!

Now approximately 3 minutes after Notes is started, Defrag.NSF for Notes will swing into action and make sure your Notes Program and Data files are optimally defragged and ready for action.

Defrag.NSF for Notes - available tomorrow.

NEW - Defrag.NSF for Notes

2011-12-05T17:23:00.001+11:00

Later this week we are finally going to release a new Defrag.NSF family member for the Notes Client.

"Defrag.NSF for Notes" is a Notes client addin that automatically defragments your Notes Program and Data directory shortly after the client starts. It will also periodically check and if the files have re-fragmented it will put them back into shape. A nice solution for an age old problem.

Here are some answers to questions we think people may have:

How much will Defrag.NSF for Notes cost ?It will be FREE for all licensed users of Defrag.NSF (our Domino Server product)

I have 20,000 clients can I install it on all those machines ?Yes, as long as you have an active Defrag.NSF (for Domino) license.

Will it make much difference to my Notes client startup times ?If fragmentation is slowing your client, then this will absolutely help. You should also note the IBM does recommend defragmenting your drive for optimal performance after a Notes Install.

Do I need to download any other third party software to make this work ?No. All the components are included.

How do I install it ?It is a simple MSI you need to run. You need admin privileges to install it. This can be scripted.

Will it work when a user is logged on with limited privileges ?Yes.

Can I continue to use Defrag.NSF without an active Defrag.NSF licenese ?No. If you don't have an active Defrag.NSF license you need to uninstall and stop using Defrag.NSF for Notes.

Does the product require a license key ?No. We trust you will do the right thing.

Why wouldn't I just use a Windows Defragger ?You have the choice. Defrag.NSF for Notes only defrags files related to Notes, so it is super quick and helps ensure you have a responsive Notes Client.

Can I schedule when Defrag.NSF runs ?Not in the initial release. This is a set and forget product.

Where do I get support for Defrag.NSF for Notes ?Since this is a free product, support will be Internet based and conducted on this Google group http://groups.google.com/group/defragnsf?src=email&hl=en
Contact Preemptive Consulting directly if you want paid Enterprise support.

Let me know if you think I have missed anything.

Changing the ACK timer can give TCP apps a big performance boost

2011-12-02T08:50:00.001+11:00

We tracked down a real pain in the bum problem where we were seeing 200ms delays for no apparent reason. Then we adjusted the Windows ACK timer and bingo... massive performance increase.

Check it out http://support.microsoft.com/kb/Q328890

Defrag.NSF features - Part 6

2011-11-30T16:19:00.001+11:00

In this instalment we will take a look at the various integrated scheduling options available.

Technical Settings (Processing Schedule Tab)

Use the settings on this tab to schedule the desired running times for defrag.NSF on this server. There are also options available to configure the defragging of transaction logs (.txn), DAOS files (.nlo) and Full Text Indexes (.ft) for this server and the option to enable the “Multi-stage” defragging Mode.

Enabling the “Multi-stage” Mode will allow Defrag.NSF to use it's multiple defragging engines to seamlessly toggle between file level and volume level defrag methods as required when dealing with difficult files. Using this feature allows Defrag.NSF to deal with insufficient freespace issues intelligently and on the-fly. When confronted with a file larger than the largest available freespace
chunk, Defrag.NSF will auto-toggle over to a full Volume Freespace Consolidation and proceed in that mode until the required amount of contiguous free-space has been assembled to allow effective defragging of the file in question.

Technical Settings (Shutdown Schedule Tab)
Use the settings on this tab to configure a defined shutdown time for Defrag.NSF. Most environments would not need to configure a Defrag.NSF shutdown time and can leave the default settings (disabled). In some circumstances configuring a shutdown time may be desirable if you notice Defrag.NSF running for extended periods on many large databases and would like to ensure that defragging does not continue to run during peak or high load times on the server.

Technical Settings (Freespace Defragmentation Tab)
Use the settings on this tab to configure the defragmentation of free-space on the nominated drive, this will correct the issue where large files cannot be effectively defragmented due to insufficient contiguous free-space on the drive. There is also an option to run this as a one-off maintenance process or as an ongoing and repeating maintenance process on the configured day(s).

Well that's it for the Tour, if you have followed this Guided Tour series of posts you will see that Defrag.NSF is actually much more than just a simple Defrag tool and offers the Domino administrator a very integrated one stop solution with a lot of features for obtaining the best performance from Domino databases and also the other drives and files on the system.

Defrag.NSF features - Part 5

2011-11-29T14:54:00.001+11:00

Continuing with the Technical Settings options we look at the ability to schedule compacts and the various selection methods.

Run Compact before scheduled defrag with option
With this option enabled Defrag.NSF will run the Compact server task with the configured Compact switch on each processed database prior to the actual defragmentation of the database. This is a handy option and can be used instead of a Program Document to run weekly scheduled Compact maintenance automatically followed by a defrag of the file to clean-up any fragmentation

introduced by the compact process. This is a perfect scenario just prior to running backup maintenance as the files will be at their smallest size and also freshly defragged providing maximum file read performance and data throughput for your backup software.

　

Selection Method

Defrag.NSF uses the selection methods configured here as “rules” for identifying which databases will be tagged for defragging during the next maintenance run. When running on a schedule Defrag will perform the Cataloging process first, during this Cataloging process Defrag.NSF checks all the databases against these Selection Methods and tags the databases that fall within the configured rules. Defrag.NSF will then perform the defragging of any files tagged via the Selection Method. If any server maintenance tasks such as Compact or Fixup were scheduled using the Technical Settings options, these tasks will be performed immediately prior to the defrag operation.

Option – When fragments exceed [n]

Use this option when you want to target for defragmentation all databases exceeding this configured number of fragments.

Option – When fragmentation Health Index exceeds [n]

Use this option when you want to target for defragmentation all databases exceeding a configured Health Index. For a detailed explanation of the Health Index, please read the previous section on the Health Index view.

Option – The top databases [n]

Use this option when you just want to always target the most fragmented databases on the server. This option uses fragment count only as the criteria for selecting the database for maintenance.

Next we will look at the scheduling options offered in Defrag.NSF

Defrag.NSF features - Part 4

2011-11-28T15:41:00.001+11:00

Let's have a look at the Technical Setting and the many configuration options available there.

The Technical Settings document provides the administrator with central access to most of the configuration and schedule settings available in Defrag.NSF.

Record before and after performance statistics

Enable the logging of performance statistics on the Technical Settings document. The Performance Reports feature makes it very easy for Domino administrators to see and document the increased performance Defrag.NSF delivers to each file it runs on. This performance equates to not only improved server performance, but also backup maintenance time reduction where files need to be accessed, read and copied during the backup process.

With the option to “Record before and after performance statistics” enabled, Defrag.NSF performs the following steps as it runs on each file.

1. Defrag.NSF performs three (non-cached) read passes on the .NSF file and logs the average read time. It then proceeds to defragment the .NSF file.

2. Defrag.NSF performs three (non-cached) read passes on the newly defragmented .NSF file and again logs the average read time.

3. The .NSF file read time is logged as a percentage improvement. The resulting stats are available for viewing in the Performance Reports view, also included are the Before/After read times and Before/After fragments for the files processed.

Performance Reports view

As described above, when the Performance statistics option is enabled, Defrag.NSF will perform “pre defrag” reads of each processed file and 3 “post defrag” reads of the file as well as the actual defragmentation run. This is a very handy feature for testing purposes to obtain before/after results and gather hard evidence of the raw performance increase Defrag.NSF can deliver to your .NSF Files and your server.

Freespace Reports view

Reports can be generated to provide valuable and detailed information about the amount of freespace for each volume, the size of the largest available freespace chunk, the total number of freespace fragments and an overall volume health indicator based on the usable state (or otherwise!) of the freespace.

Click on the “Create Freespace Report” button to create a detailed report.

Double click on the report document to open it and see the details for the drives.

The report includes the following information:

Volume: Volume Letter.

Largest freespace fragment (MB): This is the largest single fragment of available contiguous freespace.

Total freespace fragments: The number of fragments the freespace is broken into.

Status:This indicates if the largest file on the drive will fit into the largest freespace fragment. (an indicator of insufficient contiguous freespace).

If large files are not being effectively defragmented due to insufficient contiguous freespace, this issue can be resolved either manually by utilising the “Defrag Freespace” button (this runs immediately) or via the scheduling options available on the “Freespace Defragmentation” tab of the Technical Settings document. Depending on the environment, using the schedule to run freespace defragmentation during off-hours is usually the preferred method, although users and other Domino server tasks will not be affected either way.

Console output:

Next up we will continue the tour of features available in the Technical Settings.

Defrag.NSF features - Part 3

2011-11-23T15:33:00.001+11:00

For this part of the Guided Tour we will look at the different maintenance options for an individual database.

Configuring individual database maintenance options.
In any of the views you can double-click on a database document to access the Database Details, basic information about the database and also configure specific maintenance options for the database. This makes it very easy for the Domino admin to integrate the traditional Domino maintenance tasks with the scheduled defrag maintenance.

The available options are detailed below.

Exclude from been selected for Automatic Maintenance.
Ticking this box will exclude this database from being processed during the Automatic Maintenance run. You may decide to do this if you do not wish for a certain database to be included in scheduled maintenance for whatever reason.

Manually selected for defragmentation.
Ticking this box will specifically select this database for defragmentation.

Compact the database with option.
Ticking this box will set up the database for compacting with the desired option switch. If a compact option is not specified the default used will be the -b option. Immediately after the Compact completes the database will be defragged to “clean-up” any fragmentation caused by compacting.

Fixup the database.
Ticking this box allows a database Fixup to be configured for the next maintenance run. The available Fixup options are listed below once the box has been ticked.

Monitor Free Space.
Ticking this box enables Defrag.NSF to monitor the free space in the database.

Pre-allocate Free Space.
Ticking this box will enable Defrag.NSF to allocate the configured amount of free space within the database. The “Freespace Ceiling” setting is used to set a freespace (MB) amount that will never be exceeded, (This is useful when large databases are present and the percentage allocations would therefore result in excessive freespace in these large databases).

The default is 100MB but this can be set to the user's preference.

The “Pre-allocate free space” feature is very effective in preventing future fragmentation of the database by ensuring there is always a set amount of free space available within the database for additional data writes.

Once the desired options for the database have been selected, click the Save and Close button. Next time scheduled, or manual maintenance is run the configured database options will be processed prior to the defragmentation of the database.

Next up we will look at each tab of the Technical Settings and the different options available on each tab and the options for Performance Reports, Freespace Reports and the Automatic Selection Methods.

(C)opyright Preemptive Consulting - Respecting the intellectual property of others helps everyone.

Domino isn't old and clunky - but MAPI certainly is !

2011-11-23T08:26:00.001+11:00

Finding the parent (source message) of a reply document in Notes is easy, the good old $ref field. It takes one line of code.

Doing the same thing in Exchange is far more complicated, slower and involves a certain element of black magic.

Anyone that thinks Domino is old and clunky really needs to stick their head into MAPI...looks like a pile of baling twine and bubble gum to me.

Defrag.NSF features - Part 2

2011-11-22T16:44:00.001+11:00

Here's Part 2 of the Guided Tour series, looking at the Maintenance buttons.

Each view has the following buttons available to perform defrag maintenance related tasks, they include an Action, Recatalog Databases and Perform Maintenance buttons.

The Action button:

After manually selecting one or more databases, clicking the “Action” button allows the administrator to select or de-select the listed maintenance tasks.

Selected tasks will be indicated with a Green tick in the appropriate column for the databases to be processed during the next maintenance run.

The Recatalog Databases button:
The Recatalog button tells Defrag.NSF to catalog the databases on the server and create a document for each one. During this process it will also check the fragment count, database size and the Selection Method, so that any databases matching the selection criteria you have configured on the Technical Settings document can be flagged for follow-up maintenance. This will be seen as a Green tick in the appropriate column for the type of maintenance selected.

The Perform Maintenance button:
Clicking the “Perform Maintenance” button tells Defrag.NSF to recatalog the databases as described previously and then proceed with carrying out any of the maintenance tasks that have been flagged as a result of the Selection Method applied during recataloging process.

After clicking the “Perform Maintenance” button the user is presented with a dialog box with the option to proceed with the maintenance in the default manner, which includes a recatalogging of the databases, or alternatively, the user can elect to perform the maintenance without recatalogging, in which case Defrag.NSF will go straight into defragging and only the already selected databases (displaying the Green tick) will be processed. This is very useful if you have several thousand databases on the server and want to manually defrag only 1 or 2 specific databases while avoiding a complete recatalogging of all the databases on the server.

A defragging operation will always follow the other selected maintenance tasks. For example, a database that is flagged for compacting will be compacted first and then defragged to complete the maintenance.

Next in this series we will be looking at the many options available to the administrator when configuring individual database maintenance, these can include compacts, fixups and freespace monitoring/injection.

Guided Tour of Defrag.NSF features - Part 1

2011-11-21T15:10:00.001+11:00

I thought we should do a "Guided Tour" series of blog posts to have a feature-by-feature in-depth look at all the cool options available in Defrag.NSF 2.09.

Starting from the start we will work through the feature set and explain all the different options and settings available. Hopefully even long-time users will find some useful tips in this series and even new info on features they weren't aware of. We built Defrag.NSF to be a "set and forget" type product if that's how the administrator wanted to approach it, and sometimes that can allow a lot of the "hidden" cool features slip through the knowledge cracks and never get looked at. Anyway here's the Tour:

Everything can be configured from within the database itself, using the good 'ol admin client. On opening Defrag.NSF you will see the different Views available at the left side:

The Views

Each view will show the cataloged databases, each as an individual document.

By Fragments view.

The respective columns report fragment count, database path, database size and the last time the Defrag process ran on the database.

The Defrag, Fixup and Compact columns will display a Green tick to indicate which databases have been flagged for the respective maintenance task during the next Defrag.NSF maintenance run. The databases are flagged according to how you have configured the Selection Methods on the Technical Settings Document.

Health Index view.

The main feature of this view is the “Health Index” column. The remaining columns are the same as the “By Fragments” view.

What we call "Health Index" is defined as:

The number of (fragments-1) / by the size of the database * 100. The idea being that a very large database with 1000 fragments may actually be healthier (ie, less performance hit) than a really small database with 500 fragments. It allows you to focus on your best return on effort for defragging (This is useful when you are just starting out with Defrag.NSF and there is an extremely large number of databases to work on).

Utilising the "Health Index" selection method will target your most "unhealthy" databases by looking at (in simple terms) the ratio of "number of fragments/database size" and give the best "bang for buck" improvement (This is instead of looking purely at the fragments count). The approach to using the Health Index will vary in each environment but as an example, if the first cataloging of Defrag.NSF shows there are a lot of very badly fragmented databases on the server and you can see that there are 200 that indicate a Health Index of 0.4 and over, then a good strategy would be to begin by setting up the Health Index selection method to target all databases that have a reported Health Index of 0.4 and over. On the next maintenance run Defrag.NSF will target those 200 databases and ignore others, as pointed out, these are the databases that will give you the best return for the defragging effort at this stage, since they are the “least healthy”.

Once those databases have been processed you would lower the Health Index setting to then target all databases with a Health Index of 0.2 or greater, and then down to 0.1 on another subsequent run. At that point you could change selection methods over to “The Top databases” and target the top 500 databases from then on to maintain the environment. This is a good approach to take when confronted with a difficult environment and it allows the maintenance to be applied in a step-wise manner to slowly get things back into shape without wasting the time and resources required to defrag large databases with a couple of fragments in the initial stages of running Defrag.NSF in a difficult environment.

By Name view.

The first column in this view lists the databases in alphabetical order by database path name, the remaining columns are also available in the other views.

Next in the series, we'll be looking at the Maintenance buttons and what task each one allows the administrator to carry out.

(C)opyright Preemptive Consulting - Respecting the intellectual property of others helps everyone.

Warbirds - F/A18, P40, P51, Meteor, Sabre flying together - unlikely to happen again !

2011-11-21T14:01:00.001+11:00

Check out some of these photos from the Warbirds Down Under at Temora, NSW Australia on the weekend, 5 generations of 75 squadron flying together:

And some other classic old planes

Using the Lotus Notes single user to multi-user migration assistant (MUMA) tool

2011-11-18T11:33:00.001+11:00

This is a great new tool. Thanks IBM.

Abstract:

"You can use the supplied single user to multi user tool to migrate a Notes single user install to a Notes multi-user install for an existing Notes release prior to upgrading to a new release. This enables you to convert an existing Notes single user install to a multi-user install for both Notes basic and Notes standard configuration deployments. It also enables you to convert a simulated multi-user install (where the user's data directory is on a mapped drive) to a true multi-user install. This tool is available for Lotus Notes 6.5.x, 7.x, and 8.x installations on supported Windows platforms only. "

http://www-01.ibm.com/support/docview.wss?rs=899&uid=swg21459627

10 Reasons why I'll probably read your email.

2011-11-15T13:06:00.001+11:00

Recently Paul linked to 10 reasons why someone will not read a recipients email.

I'd like to suggest 10 reasons why maybe you should:

1. You took time to write to me. (if it's important to you it then might also be important to me)

2. It would be rude if I didn't (especially if you are a potential/existing customer or Business Partner)

3. Email is our primary business tool for communications

4. You might have a good suggestion

5. You might be a customer who needs help (and if you are, "my rules" about why I won't read/reply, may not mean Jack %&$# to you)

6. It only takes seconds to say 'sorry I can't help' or 'no I'm not interested' (Try it - It actually works!)

7. If you spam me, I pass that off to our spam filter - it won't happen again

8. I tend to turn off twitter, facebook and other interrupters for extended periods of time during the day

9. I only check my inbox a couple of times a day, so it tends not to interrupt my workflow

10. Its OK to collaborate.

Anyway just food for thought.

It is getting very crowded in Domino 32bit

2011-11-14T14:29:00.001+11:00

Each release of Domino adds new and exciting features, which reminds me if this graphic

But a side-effect of this is that the memory space on 32bit Domino is getting more and more crowded and the amount of shared memory Domino is pushing into all the different Domino tasks continues to grow.

When memory gets tight, we've seen situations (particularly in FP2 and FP3 of Domino 8.5.2) where Domino no longer plays nice and sometimes tasks can be terminated even if idle.

With Defrag.NSF 2.09 we've moved all Defragmenation tasks into a separate process space to ensure that memory crowding (for us at least) will no longer be an issue. It also appears to have a slight performance benefit... very nice.

For more information on Defrag.NSF please check our website at www.preemptive.com.au/defrag

Defragging FT indexes - part 2

2011-11-10T21:10:00.000+11:00

This is a follow on blog entry from yesterday's post and in it we wanted to show you how fast FT indexes fragment.

Here is a fulltext index for a user's mail file currently contains 4 un-indexed documents and is set for hourly indexing. In this case the Chronos task will update this index during it's next hourly run. For databases set for "Immediate" updating the results we see here are even more applicable.

Here we see on the console the same full text index has been freshly defragged by Defrag.NSF, it consists of one fragment, but as yet has not been updated by Chronos during the hourly run.

Then Chronos runs on it's hourly interval and updates the indexes as required, as mentioned this will include our user's fti.

Now checking a fragmented files list we can seethe indexing update has nicely fragmented bchapman.ft and now it shows up in the list in the highlighted Yellow section. The first column is the number of fragments.

Now checking the index properties, already we see 2 unindexed documents in the mail file as the user works and mail is delivered. The index will again be marked for updating and Chronos will update it during it's next hourly run, so now the cycle repeats itself.

For more information on Defragging Domino Databases, check out our site at http://www.preemptive.com.au/defrag

Defragging FT indexes - little benefit for lots of work!

2011-11-09T18:07:00.001+11:00

Many customers ask us about the defragging of Full Text Indexes and now with the 8.5.3 notes.ini setting FTBasePath the Full Text Indexes, and defragging them, seems to be back on many admins' radar. But before you go on a defragging FT campaign, you just might want to rethink if it is worth the effort - let's explain.

Full Text index files are stored under a sub-directory for a given NSF file. The contents of which are updated by the Domino Server and the indexer (Update) tasks. This update frequency is controlled by various factors and can occur at anytime from immediate to daily. When the index is updated it just about always fragments the index files. You can have a perfectly defragged index and have it re-fragment within seconds. This will keep happening over and over again.

So what effect on performance does a fragmented FT index have? Well from our testing, little to nothing, a couple of milliseconds here and there. Certainly not perceptible to the average user performing a search.

Now on top of all this, the FT files are not normally candidates for backing up so even then there are none of the backup benefits that defragging usually returns.

On busy servers with 1000s of users with indexed mail files and various other databases indexed for searching, the associated FT index files can account for many GB of data, since these files are also quite regularly heavily fragmented there is the potential for wasting significant time and server resources in defragging them for no return in the user search experience.

This appears to be a case where effort does not = reward !!

So what do we recommend you do ?

Well if you want you can get Defrag.NSF to defrag a Full text Index associated with a database, by selecting the FT Index option on the Processing schedule tab.

But alternatively the administrator could choose to schedule defragging of all the Full Text Indexes on a separate drive if the notes.ini setting FTBasePath has been implemented. That way the indexes could be optimised once a week (or even daily if you wish) and they are not going to fragment the freespace that can be better used by the NSF file

For more information on Defragging Domino Servers check out our web site at

www.preemptive.com.au/defrag

(C)opyright Preemptive Consulting Pty Ltd.

Stardock's OS/2 History

2011-10-28T11:21:00.001+11:00

I remember Stardock back in the 1990's. A great little company, interesting story. It's fascinating to hear how IBM tried to adjust course over the years. If you have a moment check out this article.

Strange things with recent contacts and 8.5.3

2011-10-28T11:00:00.002+11:00

We've upgraded a few client machine's now and hit a couple of issues we have yet to sort out:

These issues can occur on a machine:

1. All recent contacts just 'disappear'
2. Type ahead just doesn't work any more
3. Contacts that have been deleted, magically re-appear

So if you are in the process of upgrading, watch out. If we work out what is going on, I'll put the information here.

Dear IBM, it might be worth considering an Amnesty on lapsed renewals

2011-10-11T15:24:00.000+11:00

Looking at the number of accounts I now know that have very old and lapsed license versions of Domino, I wonder if it might be worthwhile for IBM to offer an amnesty period for renewals, i.e. if you renew for 12 months before the end of November we forgive you for not paying maintenance.

It might be just what's needed to bring some old customers back into the fold. Does anyone else think this could work?

Using Defrag.NSF to defrag the other drives on your server (not just the .nsf files)

2011-09-15T18:06:00.000+10:00

Defrag.NSF allows you to target and schedule other drives on your Server for defragging maintenance while Domino is running. (All done while still working in the familiar Notes interface)

As an example, on a newly built server with Defrag.NSF installed, using our "Freespace Report" tool we can generate a report that will show the current fragmentation state of the freespace on all the drives on the server.

We open the report and we decide to target the C:\ drive for a one-time defrag. (Our Domino data is on D:\ so it will be looked after with the normal Defrag.NSF schedule)

On the Freespace Defragmentation tab we can easily schedule the maintenance at a suitable time, choose the C drive and just run this one time in "Thorough" mode.

Tip: - To get an explanation of what the setting on each tab mean just click on "what's this?"

Later on the console we see the maintenance on the C:\ drive begins at the scheduled time. (we set debug logging to show detailed output)..

And the completion.

A new Freespace Report shows the improved results. (this all occurs while Domino is still happily up and running).

Cross-checking the status of the C:\ drive in Windows and it agrees we did a pretty good job!!

So there you have it, another great use for Defrag.NSF, all achieved from within Notes and without shutting down Domino. For more information go to Defrag.NSF

Compact -B isn't working on 8.5.2 FP2

2011-09-15T16:49:00.004+10:00

We have seen this a bit. You issue the compact command and the whitespace does not reduce. In particular it is a problem for mail.box databases.

There is an SPR for FP1

https://www-304.ibm.com/support/docview.wss?rs=899&uid=swg21448637

But I'm not sure if it is in FP2. Does anyone know for sure ?

To everyone that made Auslug possible...

2011-08-31T13:08:00.000+10:00

Thank you. You put on a great event and I hope those that contributed and sponsored it get many times their investment retuned.

Cheers

Ed's presentation at AusLUG

2011-08-29T21:27:00.000+10:00

Ed put on a great presentation and for the first time the penny dropped on what IBM has been talking about with the social mail client.

He demoed the client and it looked great. No longer will the backend mail server matter.

The devil is always in the detail, but if IBM can ship this (and I'm sure they can) with support for Domino/Exchange/Google etc I think we've got a winner on our hands.

For the first time in a year I clearly understand what IBM has been talking about.

Thanks Ed.

Fragmentation doesn't replicate, but new replicas always fragment !!

2011-08-18T17:30:00.000+10:00

We are regularly asked this question "Do I need Defrag.NSF on all my servers, particularly when they are clustered". The answer is a resounding "YES".

Fragmentation (and "de-fragmentation") does not replicate between databases, but replication nearly always causes fragmentation.

We recently commissioned a new server in our office and joined it to the existing cluster. After creating replicas of the databases on the new server, it is quite amazing to see how much fragmentation the process introduced and the impact it had on performance when reading or backing up those databases (The backup software must read the database first).

The screen-shot below shows the fragmented state of the database replicas on a brand-new Domino server. The other Cluster member-servers hosting the replicas of these databases have had Defrag.NSF maintaining them on an automated schedule, so they all exist on disk in an optimal state as a single fragment.

Utilising the integrated Performance Logging in Defrag.NSF, we can see there was a lot of fragmentation in the databases and there has been a significant improvement in non-cached read performance since Defrag.NSF has run and reduced the fragmentation back down to a single fragment.

Even though the other Cluster members were hosting perfectly defragged replicas of these databases, the fragmented files on this brand-new server would have rapidly deteriorated even further without the required maintenance being performed by Defrag.NSF.

You can find out more about Defrag.NSF at www.preemptive.com.au

The Lotus Zombie.

2011-07-01T14:27:00.006+10:00

The Lotus Zombie.

It could be dead, but it still feels alive.

The Lotus Zombie lives in a strange World, where double talk is normal and the truth is rarely spoken out loud.

The Lotus Zombie feels uncomfortable that organisations spend tens of millions of dollars implementing new email systems when they already have a perfectly good one. One they tend to keep for other things after the spend. It happens every day. The same organisations then complain, increase fares, hike premiums, reduce flights and cut staff. The same agencies say they have no money for the sick, homeless or unemployed.

This makes the Lotus Zombie angry - but apparently no one else cares.

The Lotus Zombie is told this is done for efficiency purposes, but wonders why any organisation would want to be less efficient!. Efficiency is the ratio of useful work divided by total work done. The output is always less, so how is this more?

The Lotus Zombie is told it is part of Modernisation, but is confused to why organisations continue to use ancient versions of the software. What have IT Departments and outsourcers been doing all these years? It is not that hard to upgrade.

The Lotus Zombie knows what Domino can do, but is bewildered that a profession based on logic, treats ignorance as an asset.

The Lotus Zombie likes to "make things as simple as possible, but not simpler", and struggles to understand why IBM is hell bent on overly complicating things.

The Lotus Zombie wonders how services companies are going to make money once everything is in the cloud.

I feel like I'm a Lotus Zombie this week. Does anyone else ?

Interesting research findings about folders from IBM's Bluemail project

2011-06-06T09:09:00.000+10:00

"People who create complex folders indeed rely on these for retrieval, but these preparatory behaviors are inefficient and do not improve retrieval success. In contrast, both search and threading promote more effective finding."

The full research paper is here

And it supports our view for the last 4 years that folders should not be a requirement for an email archiving product, see here

Is $A 30,000 too much for a single core Domino Utility Server (just the software)?

2011-06-01T17:27:00.000+10:00

or $US 32, 244. I believe that is what it costs full RRP... I think it is way too much and hope IBM looks at it when they do the xPages server pricing.

"Disk I/O as perceived by the operating system and application" - Wes Morgan

2011-06-01T15:03:00.001+10:00

Wes has written a great blog post and I love his phrase "as perceived by the operating system and application".

It is along the line of what we have been saying for years, and fundamentally why you still need to defrag your Domino data on a SAN with something like Defrag.NSF

The ID Vault may not be doing what you think it is !

2011-05-27T15:49:00.001+10:00

Some interesting things we found during ID Vault troubleshooting.

In one environment we found some issues with the ID Vault and some users who's ID had not successfully sync'd for at least 6 months, this is where you could run into problems with user's certificates expiring and never being updated in the Vaulted ID. There is no warning that sync'ing has actually failed other than a "ID not downloaded because the wrong password was supplied." message in DDM and in the Security Log, which could also mean the user has simply done just that and entered the wrong password. In a large environment with many thousands of users (often forced to change passwords at regular intervals as corporate policy) this can be a nightmare to troubleshoot because it's actually not unusual for users to enter the wrong password at times anyway.

When using the "Extract ID" feature we also found that even if the password on the extracted ID is set the same as the user's current ID password (the password on the ID in the Vault) and the extracted ID is placed in the user's local Data directory, then these ID's will never actually sync even when the user switches to the same ID (that is supposed to force an immediate sync or upload). In that scenario we would just rename the local ID and pull the ID stored in the Vault down to the user but thought it was interesting that the extracted ID's were seen by the Vault as being "different" even when the password was the same. But again, if the local ID has been deleted or got corrupted and Vaulted ID has not sync'd for the last 6 months and the certificates in the ID are expired then you are back to square one with a user who cannot access the server.

Another interesting find was that when checking the local user ID with the File -> Security -> User Security dialog we found that if the ID file had at some time in the past successfully uploaded into the ID Vault there was a message indicating "This id file has been backed up into the vault" even if the Vaulted ID and the Local ID were not currently sync'ing correctly(and hasn't for some time). This created another issue because in the case of a corrupted or deleted local ID, the ID in the Vault could not be pulled down because the password stored on that ID was 5 previous passwords old (not sync'd for 6 months) and the user either could not remember it or was not allowed to reuse it now. This requires a Password Reset for an operation that should really be seamless and not require admin or user interaction.

Oh yeah!, don't get me started on the whole Password Reset self service thingo you know, the one where the user logs on to the server over http and can Reset their forgotten Notes Password after authenticating with their http password that was probably the same as the Notes one they've forgotten.............

A Query for the ID Vault Experts out there !

2011-05-24T17:33:00.001+10:00

ID Vault Challenge question !

If the Administration Process is used to recertify a user with a vaulted ID file, do the certificates in the user's ID file held in the vault get updated automatically with the new certificates or is interaction with the Notes client still required?

For example, if the user does not log on and authenticate with the server until after the certificate expiry on the local ID file, will the ID in the Vault contain the new certificates and allow authentication if the expired local ID is deleted?

Comments please.

5 Reasons you need Defrag.NSF

2011-05-12T16:06:00.000+10:00

Here's 5 reasons you should get your hands on Defrag.NSF:

1. Your backup times are hideous - Defrag.NSF typically improves backup times by 40%
2. Intelligently targeting problematic databases - only defragging databases that need to be done. Sometimes trying for zero fragmentation can be expensive.
3. Improved algorithm for super large Domino databases
4. Slow view rebuild times - by removing fragmentation view rebuilds will be faster resulting in a snapper user experience
5. Improved set and forget administration from within Domino

Version 2.06 is now shipping, better, stronger, faster.... check it out, Get Defrag.NSF now

Is the new name for Lotus Software, "IBM Collaboration Solutions" ?

2011-04-13T12:08:00.000+10:00

I've seen a few people within Lotus with a new 'Brand Name' of "IBM Collaboration Solution"

Interesting.

Not all defragging engines are equal

2011-03-31T18:19:00.000+11:00

I've written about this before, but it has come up a couple of times this week - Not all defraggers are the same.

Some are free, some cost money, some have very intelligent algorithms, some aren't so good. Tweaking and optimising a defragging algorithm takes time. Time typically has a cost associated with it.

One thing we've learnt over the last three years is that Domino places some rather unique requirements on a defragmention algorithm. The following rules seem to always apply:

the longer the server has been running, the worse state it is in.
available freespace is likely to be highly fragmented.
it is not unusual to have highly fragmented multi gigabyte database. As a rule, multi gigabyte files (30GB+) outside of Domino are fairly uncommon.
it is difficult to defrag files that are constantly changing.

In an environment where you have a lot of small databases with a lot of disk space available, the job of a defrag engine is rather easy.

As you move up the scale, and enter the real world of Domino it gets harder and harder, and the algorithm used by the defrag engine needs to be smarter.

I've been asked a number of times, why not just use the 'Free stuff'. Free defragging products typical are:

not designed from the ground-up to meet the needs I outlined above.
perform relatively simple cluster moves.
do not handle multi-gigabyte, constaintly changing files very well.
are a simple wrapper around a freeware utility that falls in one of the categories above.

I'm not saying they are bad, or don't work, but what I will say is this - developing a Domino specific defragging engine has been challenging, very time consuming and costly.

Which brings up a well known idiom, "You get what you pay for"

Given all this, we are please to announce the availability of Defrag.NSF version 2.06. We have done a significant amount of fine-tuning to the defragging engines for this update and we are excited about the results we're obtaining.

Put Defrag.NSF to work on your servers today with a 30 day, supported Free Trial and see for yourself. Register here

IBM agrees with us that Domino on a SAN's is affected by fragmentation.

2011-03-23T09:04:00.000+11:00

As noted in their "Optimising Lotus Domino Administration" Redbook.

Check out Defrag.NSF for more information.

Monitoring Archival Transaction log space to prevent a server crash

2011-03-22T05:20:00.001+11:00

The following is a great suggestion from Grant.

If you are using Archival-Style Transaction Logging and the Transaction Logs are not backed up frequently enough or if the backup solution fails and does not backup and flag the backed up log extents as free, this can lead to the drive filling up entirely and crashing Domino.

Here are some steps for how to set up an Event Generator monitoring for low disk space on your Transaction Log drive and then to send you an email so you have a chance to sort something out before it fills up the drive and crashes your server. For example, to monitor your E:\ drive you would do the following steps.

Open events4.nsf (Monitoring Configuration) database and open the "Event Generators" -> "Statistics" view, the click the "New Statistic Event Generator" button.

On the Basic tab, select the server to monitor and point to it from the drop-down list.

In the "Statistic to Monitor" section, choose "Single Instance Statistic" and select your drive letter from the drop-list. Choose the "Monitor as a number of bytes" option. See pic:

On the Threshold tab set it up as below, this will monitor for approx 5 Gig of space, set it to whatever you are comfortable with.

On the "Other" tab click the Create a new event handler for this event" button.

Click OK to save the Event Generator when prompted. The Handler Wizard will start up, click next and select the "Mail" option.

Select yourself from the drop-down list to receive the email, then click through to the Finish of the Wizard.

That's it! You will now have a corresponding "Event Handler" for that event and if you click on the "By Action" view under Event Handlers you should see a Mail action set to send you an email if ever the E:\ drive gets down to 5 Gig of space, and you'll have a chance to get the problem corrected.

Lotusphere Comes To You renamed as Collective Intelligence. Is this the start of a trend ?

2011-03-09T11:56:00.000+11:00

IBM did this a few years ago in Australia.

Now it is interesting to note that on the Agenda and landing page for the Conference that Lotus, Notes and Domino are really not mentioned. Wow.

http://www-07.ibm.com/events/au/collectiveintelligence/

After spending so much money on Lotus Knows branding I am suprised. I wonder if this is happening elsewhere?

Last day to take advantage of the Defrag.NSF special

2011-02-28T16:20:00.000+11:00

It's your last day to take advantage of our current Defrag.NSF special. For all the information click here

My ingredients list for a "Domino Social Server"

2011-02-19T09:36:00.000+11:00

There has been a lot of talk about a Mac admin and designer client, but how about if IBM created the "Domino Social Server"

Here is my ingredients list:

- get one Domino 8.5.2 Server then add
- Sametime for Domino blend with
- a new xpages version of the Notes Admin Client add
- a big dollop of xpages versions of everything in Lotus Connections. Put in
- a sprinkle of Symphony live hosted on Domino and
- a dash of Vulcan to merge disparate inboxes

Presto, you have a fantastic "Social Server"

It is just so simple, I'm not sure why someone hasn't done it!!! LOL

The Turbo encabulator makes more sense than MAPI

2011-02-19T09:23:00.003+11:00

Watch this video and you will know what I am talking about !

Special Lotusphere pricing for Defrag.NSF has been extended until the end of Feb

2011-02-15T14:07:00.000+11:00

We have decided to extend our offer until the end of Feb 2011. For all the details please check our website

That is one BIG Cyclone

2011-02-02T15:56:00.000+11:00

Poor Queensland is about to cop it again.

http://www.news.com.au/breaking-news/floodrelief/north-queensland-braces-for-cyclone-anthony-as-cyclone-yasi-brews-behind-it/story-fn7ik2te-1225998711771

This thing is massive !!

http://www.news.com.au/breaking-news/floodrelief/how-cyclone-yasi-compares-around-the-world/story-fn7ik2te-1225998762870

Stay safe

How to deploy Defrag.NSF to 400 servers in 6 clicks

2011-01-13T20:18:00.000+11:00

Deploying a product can be time consuming, especially when you have a lot of servers. Some defrag products require downloading other exe's, copying files and updating Notes.ini files on the servers. This process can get very time consuming. If your time is worth money, you don't want to be doing that.

To help customers deploy, upgrade and monitor Defrag.NSF we created the Defrag Enterprise Console.

Here is an example of how you would use it to deploy Defrag.NSF to 400 servers.

You can find out more about Defrag.NSF at our Web site here

Victorian Lotus User Group Presentation

2010-12-10T13:03:00.000+11:00

Last night I had the privilege to present to the Victorian Lotus User Group.

Here is a copy of what we talked about.

Special Lotusphere pricing on Defrag.NSF V2

2010-12-06T17:50:00.001+11:00

To celebrate Lotusphere 2011, we are offering Defrag.NSF V2 for $100 per server, which includes full technical support and upgrades for 12 months.

This release has some amazing new features that we're sure you'll like, these include:

Still easy to install, it's a Notes database! (nothing to download, nothing to edit - and it's fully supported!)
A completely new defragmentation engine that can defrag multi-gigabyte databases in seconds.
The ability to defrag databases where there is very little freespace.
A new multi-stage defragmentation process that automatically switches between file and volume level defragmentation when required.
Freespace consolidation.
Integration into the Defrag.NSF Enterprise Console (available on request) for widespread 1-click deployment, mail-in reporting and upgrade automation.
1-click generation of Freespace reports showing the largest contiguous Freespace area on your disks.
Enhanced integrated scheduling options.
Perform all the above tasks from the familiar Domino Administrator Client user interface.

If you like more information please check our website

File or Volume defrag! What's the best?

2010-12-03T03:17:00.000+11:00

Often people have asked us about the benefits of Defrag.NSF over any of the commercially available "Volume" style defraggers, and some have even been surprised when we showed them how the free defragger they were using wasn't actually doing as well as they though it was.

That is always the easy part for us, as the back to back results and the straight forward flexibility and configuration options available in Defrag.NSF just leave no doubt, it's just too easy!

The real answer is that it all depends on what you want to do.

What I do know is that there are a lot of different ways to defragment a Volume. Some common methods include; systematically defragmenting all the files, reordering the files, consolidating the freespace as tight as possible, consolidating freespace wherever possible. The list goes on.....

Volume defragmentation can be I/O expensive, particularly on a Domino volume that is fragmenting rapidly due to data change, not only that but it is not always the best approach to dealing with fragmentation on a Domino server, for example a completely defragged volume with no free-space allocated to individual files will re-fragment the individual files quite quickly. Defrag.NSF's pre-allocation of freespace can help here and this is one big advantage of Defrag.NSF, an advantage that is almost always overlooked by people using "Volume" type defraggers. In fact it becomes a vicious cycle requiring inefficient full Volume defrags at overly frequent intervals - not ideal at all!

Surprisingly, there are also a number of different ways to defragment an individual file, but most file defraggers will just simplistically look for a freespace block big enough to hold a file and move it there.

So what do we do in Defrag.NSF?

Defrag.NSF is able to intelligently utilise both File and Volume defragmentation and switches between the two seamlessly as required to achieve the desired result, again here is a big advantage over traditional "Volume" derfaggers. Where possible, Defrag.NSF tries to do a file move, if the file is large, Defrag.NSF analyses the MFT, works out where its fragments are, and then reassembles them as efficiently as it can. This feature allows the product to defrag multi-gigabyte files quickly.

If there still isn't enough freespace, Defrag.NSF can then consolidate the Volume until there is. The system does this automatically and intelligently.

So File or Volume defrag? What's the best? ... Let Defrag.NSF decide...

iNotes Active X Installer

2010-12-01T07:48:00.000+11:00

It has been a long time coming. iNotes 8.5.2 now includes an installer so you can pre-install the iNotes ActiveX controls. You can find it in the data\domino\html directory.

I am still searching for documentation on it, and will post details when I find it.

Calling for people who are willing to provide feedback on Defrag.NSF for Notes

2010-11-29T17:42:00.001+11:00

Defrag.NSF for Notes - yes the client version !

2010-11-18T07:47:00.000+11:00

We've taken a lot of what we have learnt in developing Defrag.NSF for Domino and put it into a new native Lotus Notes solution that's never been done before.

The Notes 8 client can consist of over 18,000 files. During installation the installed files will fragment all over the PC's hard disk slowing down Notes startup times and inhibiting the user experience while using Notes. The Notes databases accessed by the client are certain to become more fragmented over time and this will also further degrade performance, eventually resulting in relatively sluggish response from a user perception.

So to help solve these two issues, we've taken the Defrag.NSF engine and created a Notes client addin that calls it shortly after Notes starts. This ensures that the files Notes needs are in optimum shape from that point on. Our tests show excellent performance gains, and IBM recommends performing this sort of maintenance in their wiki.

We are seeking people interested in testing this new offering and who are willing to give us constructive feedback on what they find. If you are interested, drop me a line at info@preemptive.com.au.

Defrag.NSF for Notes, coming soon.

It's Here - Defrag.NSF V2, a good thing just got a whole lot better.

2010-11-17T15:29:00.000+11:00

After a year of development we are pleased to announce Defrag.NSF Version 2 is now available!!

This release has some amazing new features that we're sure you'll like, these include:

Still easy to install, it's a Notes database! (nothing to download, nothing to edit - and it's fully supported!)
A completely new defragmentation engine that can defrag multi-gigabyte databases in seconds.
The ability to defrag databases where there is very little freespace.
A new multi-stage defragmentation process that automatically switches between file and volume level defragmentation when required.
Freespace consolidation.
Integration into the Defrag.NSF Enterprise Console (available on request) for widespread 1-click deployment, mail-in reporting and upgrade automation.
1-click generation of Freespace reports showing the largest contiguous Freespace area on your disks.
Enhanced integrated scheduling options.
Perform all the above tasks from the familiar Domino Administrator Client user interface.

Try it for your self by registering for a free fully supported trial

Why is finding a message by message-id so hard in Outlook ?

2010-10-30T08:02:00.000+11:00

In Notes you can do a simple db.search, e.g.

Set collection=db.search({$messageid = "}+ strMessageID+{"}, Nothing, 0)

Doing the same thing with Outlook requires you to locate the IPMRootFolder folder, then you need the recursively search each of the subfolders until you find the message.

Certainly not as easy, and my guess is it takes significantly more processing time.

Defrag.NSF V2 - Multi-stage Mode

2010-10-29T17:13:00.011+11:00

As noted in our previous blog articles, sometimes due to the free-space fragmentation on a drive, certain large databases may not be completely defragmented if they are larger than the largest free-space chunk. This is not an issue for Defrag.NSF V2 because it utilises multiple defrag engines which can be automatically switched "on-the-fly" for the job at hand.

A dedicated "one-off" full volume consolidation can be one possible answer here, and this is the approach a less sophisticated product will require, and Defrag.NSF can be configured to use that approach if desired, but there is also a better way.

Enter Defrag.NSF "Multi-stage" Mode

Enabling the “Multi-stage” Mode will allow Defrag.NSF to use it's multiple defragging engines to seamlessly toggle between file level and volume level defrag methods as required when dealing with difficult files. Using this feature allows Defrag.NSF to deal with insufficient freespace issues intelligently and on-the-fly.

When confronted with a file larger than the largest available freespace chunk, Defrag.NSF auto-toggle over to a full Volume Freespace Consolidation and proceed in that mode until the required amount of contiguous free-space has been assembled to allow effective defragging of the file in question. Defrag.NSF will then intelligently switch back to file-level processing.

SPECIAL OFFER !!!!

Take advantage of our special Lotussphere offer or register your details and enter V2 in the coupon code today and we will forward you your trial version this week.

Defrag.NSF V2 - coming soon, the most intelligent, innovative and effective Domino specific defrag tool EVER!

Defrag.NSF V2 Walk Thru

2010-09-30T17:07:00.000+10:00

Here is a video walkthru of the feature rich Defrag.NSF V2.

Available for on the 12th of October 2010.

Why the Group Calendar might show "info restricted" incorrectly.

2010-09-29T16:52:00.000+10:00

We've just been working on a site where a number of users had the Group Calendar showing their scheduled information as "info restricted".

Looking at their mail preferences, we could see that a mail policy had set their schedule access to allow "everyone" to see their schedule information. This was locked to prevent change. Everything looked OK.

We "back out the policy" and then selected the "Only these people and groups" option. At this point names that had previously been configured appeared.

Removing these names magically corrected the problem and the system began to work.

Hopefully this information is going to save someone a truck load of time.

Two reasons why the World's largest Domino Sites love Defrag.NSF

2010-09-29T11:49:00.001+10:00

1. It works and delivers on its performance promises.

2. It is very easy to install. There is nothing to download or ini files to change. Just create the database, press the install button and add the license.

Just to show you how easy this is, we've created this short video on how to install the product.

If you'd like to know more, request a trial

What's new in Defrag.NSF Version 2 ?

2010-09-10T15:26:00.000+10:00

Centralised Configuration - Technical Settings:

While the Technical Settings document is not a new feature, Domino Administrators will love the easy access to the new features and the easy configuration options available using the Technical Settings document.

The General Settings tab - new feature:

Mail-in address for Defrag alerts, this is where the Domino admin can nominate a central mail-in database to receive Defrag reports. This feature will provide enhanced visibility for those Domino administrators who have deployed Defrag.NSF throughout a large environment, sometimes 100's of servers, and need to easily have an overview of any existing issues on the servers that may otherwise be overlooked.

Processing Schedule tab - new feature:

On this tab the administrator has always been able to set a suitable schedule to best suit the environment, now there are options to also include your Transaction Log files (.txn) and DAOS files (.nlo) in your defrag schedule.

Freespace Defragmentation tab - new feature:

Use the settings on this tab to schedule the defragmentation of free-space on the nominated drive, this will correct the issue where large files cannot be effectively defragmented due to insufficient contiguous free-space on the drive. There is also an option to run this as a one-off maintenance process or as an ongoing and repeating maintenance process on the configured day(s). Why would you want to defrag freespace? If you don't know the answer to this question, Read our blog on this subject here

Freespace Reports - new feature:

Here's an example of a Freespace Report:

As we just saw above, you can defrag the freespace on selected drive via a schedule or, here you can choose to defrag the freespace manually with the "Defrag Freespace" button:

Version 2 is due for release very soon, but why wait any longer? Upgrading from the current version to the new version is just a design replacement.

So go to our website and register NOW for the current version of Defrag.NSF, and start using it, to stop fragmentation creating unnecessary I/O and burdening your Domino servers, slowing your backups, slowing response times and everything else your server needs to be doing instead of waiting at a Red Light because the Operating System thinks your .nsf files, .nlo file and txn files are fragmented.

Get Defrag.NSF here

All existing customers will receive free upgrades to Version 2.

A closer look at Smart File Defragmentation

2010-07-29T15:20:00.010+10:00

In my previous blog post I mentioned the "Freespace Consolidation" and "Smart File Defragmentation" components in Defrag.NSF V2. So, how smart is Defrag.NSF V2 and what will the end result differences be when comparing the kind of intelligent strategic file placement Defrag.NSF V2 uses, compared to simply using a freely available tool like contig.exe?

We demonstrate this in the pics below, the difference is in fact quite remarkable in the final result, between each of these runs the server was restored back to the same fragmented starting point each time.

Remember when viewing these pictures that:

- Red bars are bad.

- Small Blue bars are good.

- Large Blue bars are great

- Large White bars are what we are after along with Large Blue bars.

Comment - So in the beginning the volume is a mess.

Figure 2 - After contig.exe has completed its run.

Comment: Files are nicely defragged, but freespace is a mess. New new large files will fragment.

Figure 3 - After Defrag.NSF V2 completed in "Fast" mode.

Comment: File are nicely defragged and freespace is very usable.

Figure 4 - After Defrag.NSF V2 completed "Freespace Consolidation" mode

Comment: Perfect ! Please note small sliver of white space is actually the MFT !

Defrag.NSF V2 -Why freespace defragmentation is also important

2010-07-28T17:06:00.006+10:00

One of Defrag.NSF's primary functions is file level defragmentation. This is because a Domino database typically represents a single file on disk.

One of the historical issues with file level defragmentation products is that they need a large enough freespace block on disk to move the file into. This can get a bit tricky when people have large (think 25GB+) mail files.

Defrag.NSF version 1, uses a number of different techniques to perform this relocation, including the ability to move to multiple freespace holes if a suitably large enough one did not exist. There is however a chance that even though there is plenty of freespace on a volume, it just may not be usable.

Let look at an example. In figure 1, there is plenty of freespace, and it consists of two very large contiguous sections. This is a great scenario.

Figure 1.

In figure 2, we have the same amount of freespace, but from a classic file defragmentation point of view it is basically unusable for large database defrags. The freespace itself is fragmented.

Figure 2.

So to help out with this exact situation, Defrag.NSF V2 is going to include two new features:

1) Freespace Consolidation - the ability to pull back all the freespace into a single chunk so it can be used, in effect, defragmenting the freespace

2) Smart File Defragmentation - the ability to intelligently defrag a file by strategic relocation of other nearby files, and allowing optimum usage of the freespace adjacent to the original file's current location.

Over the next two weeks I'll step you through how these features work as we prepare to release Defrag.NSF V2.

Defragging NTFS on a SAN DOES make a difference

2010-07-27T12:19:00.003+10:00

I noticed in one of my previous posts, a comment on the fact that because SANs typically fragment their data physically over multiple disks, then they are "fragmented by design" and hence do not benefit from being defragmented.

Unfortunately the author of that post does not understand the difference between defragging the SAN and defragging the NTFS volume on the SAN.

In simple terms, if NTFS is fragmented, Windows has to issue more I/O requests (spilt I/O etc). These I/O requests added up very quickly and slow the overall performance of the OS and the SAN. In this scenario, it is Windows perception of the disk that matters. It spends a significant amount of time blocking and waiting on I/Os that just aren't required.

At the end of the day, the evidence also points to the fact that defragging makes a difference. Time and time again the results are the same. The clock does not lie.

If your SAN administrators tell you otherwise, then ask them to do a test... but be careful some people don't like to be proven wrong.

If it's your Domino server... give Defrag.NSF a try.

Lotus Knows suggestion - the Swiss Army Knife approach

2010-07-26T12:57:00.003+10:00

Do you think it would work ? I think it's a winner !!!

File fragmentation of 98% appears to be common

2010-07-23T15:37:00.000+10:00

I've just done some sampling on a number of Domino 8.5.1 servers that have not be defragmented since they were build (2+ years) and it appear that 98% file fragmentation is rather common.

Further proof that when left to its own, NTFS and Domino can make a rather nasty mess.

So if you have never done a Defrag, startup the Windows defragmentation tool, point it to your Domino data volume and run an analyse. Let me know what you find.

If it is a nasty mess, give Defrag.NSF a try.

Mail Database fragmentation over time

2010-07-22T16:32:00.004+10:00

We've been busy working on version 2.0 of Defrag and some of its awesome new features (more on that soon) and during this testing process we wanted to find out just how fast fragmentation occurred on a perfectly clean Domino Server.

We used IBM's testing and simulation tools to represent 200 users sending and receiving mail over a 7 hour period. We stopped ever hour and measured the results, which confirmed things can get rather bad, very quickly. Please see the graph below.

If you'd like to know more about Defragging Domino databases, please see our website and register for a FREE trial.

Updating bookmarks with policies appears to be broken

2010-05-25T10:59:00.000+10:00

We've been revisiting updating bookmarks via Desktop policies and it appears that in 8.5.1 it is (still?) broken.

When the policy is initially applied some users get the bookmark, some do not. We have tried the debugging steps outlined in IBM's Troubleshooting policies and settings document. These steps sometimes work but more often than not don't.

Interestingly in the 8.5.1 contacts database an number of the agents that the technote tells you to run no longer exist.

Another interesting feature is that if a user gets a bookmark icon added, and then they deleted it, there appears to be no way to add it back via a policy.

Has anyone actually got bookmarks and policies working together nicely ? If you could point me in the right direction I'd greatly appreciate it.

New Defrag.NSF videos posted

2010-04-23T13:56:00.005+10:00

Defrag.NSF has evolved nicely over the past year and it was time to update our videos of it in action.

If you haven't used a recent version of Defrag.NSF I'd like to suggest you take a quick look. I think you might like what you see.

Single Database Defrag
Busting the Myth (of Compact as a defrag tool)
Scheduling Automatic Database Maintenance

Lotus Knows is coming to Australia

2010-03-25T18:52:00.002+11:00

I got word today that Lotus Knows will be kicking off in Australia and NZ late 2nd quarter for a full launch in the 3rd Quarter. This the best local news on Lotus I have heard for years.

Thanks for listening to us IBM, now we might be able to turn the tide. Only Lotus Knows.

Allianz drops Lotus mail for Outlook

2010-03-18T16:38:00.002+11:00

As reported today in the news. Possibly old news really.

IBM must bring Lotus Knows to Australia sooner than later. There are more large customers here on the tipping point. Face it, the current strategy is not working.

Want more disk space? Defrag.NSF can help you!

2010-03-04T18:16:00.008+11:00

In a previous blog, let's call it Part 1, I showed how utilising the combined power of DDM and Defrag.NSF provides a valuable resource for the Domino administrator and can be used to manage file free space on the Domino server. Using this targeted approach can actually provide some astounding disk space recovery results. If you're wondering how this approach shakes down on a production server, here's the results from using Defrag.NSF and it's integrated tools overnight on a customer's production server. We used the approach outlined in Part 1, configuring Defrag.NSF to recover the disk space from just 2 of the databases our DDM probe identified for us.

Using Defrag.NSF in this manner on just these two databases gave us 2 GB of disk space back and this is without even getting serious.

Here's what we had to start with:

Here's the first database before Defrag.NSF performed scheduled maintenance (Note the size, fragments and free space):

Here's the second database before Defrag.NSFperformed scheduled maintenance (Note the size, fragments and free space):

Now, here's the first database after Defrag.NSF performed scheduled maintenance overnight (Again, note the size, fragments and new free space allocation):

Here's the second database after Defrag.NSF performed scheduled maintenance (Note the size, fragments and new free space allocation):

Here's the disk space we ended up getting back, 2 GB!! (up from the 34GB we started with).....That was almost too easy, ...and we weren't even trying!!

So there's a brief look at the power of Defrag.NSF at work on a customer's production server, we come up with three very desirable results:

1. Over 2 GB of recovered disk space.
2. NSF files in a single contiguous fragment.
3. A configured amount of free-space in the files for new data, preventing further fragmentation.

As with Notes providing more than just email, Defrag.NSF provides more than just defragging.

MAKE_DOMINO_BACKUPS_GO_FASTER=1

2010-03-03T14:58:00.004+11:00

While you can try the MAKE_DOMINO_BACKUPS_GO_FASTER=1 setting to punch your Domino backups through quicker, it's guaranteed you'll be disappointed with the resulting improvement. Domino administrators everywhere are finding the best solution to this issue is to simply schedule Defrag.NSF to run each night before the scheduled backup time to ensure the fastest possible transfer of data from Domino to the backup media.

Schedule a Defrag.NSF start time for each day:

If desired also schedule a Defrag.NSF shutdown time for each day:

Here's a sample before/after scenario obtained from one of our customers backup logs. This sample is the result of using the Defrag.NSF scheduling interface in the Domino Administrator client to configure Defrag.NSF to run on the server in question each night for a few nights while the administrator was at home. "Data throughput rate" results were then obtained from the backup logs. An improvement of what equates to 12Gig per hour extra data pushed across to the backup media is the kind of result any Domino administrator or Backup administrator would like to see.

Before Defrag.NSF:

After Defrag,NSF:

Defrag's Database Health Index

2010-03-02T16:34:00.005+11:00

One of the unique features of Defrag.NSF is the "Health Index" view.

We are often asked "What is the Health Index used for?". What we call the Health Index is a method we can use to define the health state of a database using the relationship between the actual size of the database and the number of fragments currently in the database. When a server is in a particularly poor state with many databases being very fragmented, some of these database may also be very large and take relatively much longer to process, so we can use this method to harvest the most productive gains early in our defragging process. The idea being that a very large database with 1000 fragments may actually be healthier (displaying less read-performance hit) than a really small database with 500 fragments.

Essentially it gives the Domino administrator a good "first step" strategy for defragging a badly messed up server in the first instance by allowing you to focus on your best return on effort for defragging (This is when there is an extremely large number of databases to work on). Utilising the "Health Index" selection method will target your most "unhealthy" databases by looking at (in simple terms) the ratio of "number of fragments/database size" and give the best "bang for buck" improvement. (This is instead of looking purely at the fragments counts).

Once the Health Index view of the databases has been examined, a suitable number can be configured for the Selection Method and when the defrag process has run across these databases, you can then "dial-down" your configured "Health Index"setting in stages and thereby gradually work through defragging all the databases on the server in a series of "chunks", getting a more effective and faster performance return for the work done. Once all the databases have been processed you can change the method to "the top databases" with automatic allocation of free space to maintain the new level of performance.

Centrelink's new website - built on Domino, very nice

2010-02-26T07:55:00.003+11:00

Centrelink is a huge Australian government department that has been using Domino for their Website for years. They recently gave the site a facelift. Check it out.

It is most likely the highest profile, most widely hit, Domino website in Australia.

Nice work.

Using the combined power of Defrag.NSF and DDM

2010-02-25T17:11:00.010+11:00

Defrag.NSF and DDM are both powerful tools for the Domino administrator. Here's a neat way for a Domino administrator to combine the power of these two great tools to help minimise "wasted" free space by reducing file size in targeted databases on a Domino server. You can use a DDM Database/Scheduled Checks probe to alert you when the amount of unused space in a database is beyond a configured threshold. In the pic below, files in the mail\*.* directory will be monitored for wasted space beyond 1/3 the total size of the database, this valuable disk space can be better used somewhere else on your server.

Now, you will notice there is an option just underneath where you can "Automatically compact the database", the problem with selecting this option is that you cannot specify what style of compact you want and when you have transaction logging enabled the default -b option will be used and there will be no file size reduction, that is where Defrag.NSF will come into play.

Once this DDM probe runs, any database exceeding your threshold setting will trigger the following type DDM document:

With this information we can target the database(s) in question and configure Defrag.NSF to run a Compact -B at the next scheduled maintenance run (yes, the database is small for demo purposes, but you get the point, and it also has one fragment because Defrag.NSF is maintaining this server). This will happen as scheduled without further involvement from the administrator, here are the settings before the maintenance has run.

Now Defrag.NSF will perform this maintenance and defrag the database as an integrated operation as defined by your configured Defrag.NSF schedule:

Here is the result, notice the database size and the number of fragments after the maintenance, compared to the before:

If transaction logging is enabled, best practice dictates you should backup soon after running this style of compact. (DBIID changes)

So there is a very useful way to use these two tools together and harness the power they both offer the Domino administrator, the disk savings are very impressive when this approach is used on even just a few large files with a lot of trapped space. It doesn't take very long at all to set this up and it's a nice surgical approach delivering good results compared to just running Compact on everything in sight (shotgun approach).

How to Prevent NSF fragmentation by taking "Preemptive" Action

2010-02-24T18:16:00.005+11:00

There's an old saying, "Use the right tool for the job", and the job I'd like to discuss is adding free space to a Domino database that has been freshly defragged and in a contiguous state on disk. Adding free space to a freshly defragmented .nsf file is a smart and "Preemptive" way to help stave off further fragmentation of that file when new data needs to be written into it.

What you are doing in effect is "reserving" a place for that new data so that it will be written along side the existing data in a contiguous state and not written somewhere else on the disk that will require split I/O every time it needs to be accessed, and again when it is time to back that data up.

The right tool for the job in this case is Defrag.NSF, using the option to "Monitor Free Space". When the administrator selects this option and configures the free space desired for that particular database, Defrag.NSF will monitor that setting for the database and make sure that amount of space is always available for new data to be written. This happens during the automatic defrag maintenance schedule and if this setting is sufficient your nsf files will never fragment again.

The beauty of Defrag.NSF is that this setting will be continuously monitored, and any fragmentation that does occur due to data "overflow" will be promptly and automatically dealt with at the next scheduled Defrag.NSF maintenance run, along with the fresh allocation of the free space.

For more information check out our website at www.preemptive.com.au

Here is an example of a Database settings document

Getting Lotus Knows to Australia

2010-02-19T11:30:00.002+11:00

Apparently the Lotus Knows campaign running in America is doing wonders for Lotus in the USA. We're also told there is going to be a World Wide roll out and Germany is next. But who is next after that ?

To help push ANZ up the list, what I'd like to suggest is that any Business partner or Customer in Australia takes 5 minutes and writes an email to "Glen Boreham" the Managing Director of IBM Australia and New Zealand and asks in the nicest possible way that he requests Lotus Knows and supports this as a local campaign.

The IBM website tells me that Mr Boreham's email address is glenb@au1.ibm.com

Asking can't hurt, just make sure you do it in a nice and pleasant way.

Anyone know how to force an ID into the ID Vault ?

2010-02-03T11:01:00.002+11:00

Rather than wait hours or days for it to happen. Is there anyway to make it happen straight away for a user ?

Just let me know. I will post details if we work it out.

Lotus results are down 5% in the 04 Qtr 2009

2010-01-21T13:03:00.002+11:00

All the info is at

http://www.ibm.com/investor/4q09/press.phtml

Lotus knows we need Lotus knows World-wide sooner than later

The results speak for themselves:

0408 - Flat

0109 - Lotus down 12%

0209 - Lotus down 2%

0309 - Lotus down 9%

0409 - Lotus down 5%

Easter Eggs are already in the shops !

2010-01-13T14:04:00.001+11:00

and hot cross buns... any get the feeling this year is going to be over before it begins ?

The winners are in...and it was a 6000% improvement

2009-12-07T15:25:00.007+11:00

Our winner reported a spectacular improvement of over 6000%...

So, just imaging what Defrag.nsf can do for you.

Congratulations to our winners who will be enjoying their new iPods before Christmas. Thanks again to everyone that sent an entry in. Your support is greatly appreciated.

The iPods are ready - so where the bloody hell are you !

2009-12-03T09:51:00.007+11:00

The iPods look fantastic, and with only one day to go, it is time to get those entries in.

www.preemptive.com.au/competition

Oh and if you wondered about the title check out what Lara had to say

Anyone seen any research on what mail client's people use at home ?

2009-12-02T14:49:00.002+11:00

I recently had an exec say to me that most people use Outlook at home therefore it is better that they use it at work.

I'd think most people actually use some form of web mail at home... anyone seen any research on this topic ?

This might help with the Afghan problem

2009-12-02T14:12:00.003+11:00

90% of the World's Opium crop grows there... I wonder where they get the money....

Only 3 days left to enter our Defrag.NSF competition

2009-12-02T09:20:00.002+11:00

Only three days to go. Get your entries in soon. Click here for all the competition details

Smart upgrade run as does not work for the 8.5 -> 8.5.1 Multi User client

2009-11-25T10:41:00.002+11:00

After a week of trying, we can confirm that SURUNAS does not work when you attempt to upgrade from 8.5 to 8.51 Multiuser.

You get multiple error messages and it appears the only work-around is to give users Local Administrator access prior to performing a normal Smart Upgrade. Power user access is not enough, it must be Local Administrator access. This can be done with a Group Policy Object in AD.

Please note that this information relates to attempting to upgrade a 8.5 multi-user client to 8.5.1 multi-user. If you have different clients your milage will vary.

Did you know that Defrag.nsf is powered by Backup Booster !

2009-11-25T10:39:00.000+11:00

What is Backup Booster? Backup Booster is our internal code name for the engine that Defrag.NSF uses to move file around.

Why do we call it Backup Booster ? Simply because that is exactly what happens when Defrag.NSF has finished its run. Your Backups are faster and you will have significantly less I/O. You will have that 'new system feel'.

Find out why customers all around the World are deploying Defrag.NSF for yourself, and give your backups a boost while you are at it.

Give Defrag.NSF a try, and while you are there enter our iPod Competition.

Anybody ever got Smart Upgrade Run As Utility to work.

2009-11-19T06:09:00.003+11:00

We have been trying for a while to get this utility to work so that non-power users can upgrade their client but it appears to be broken.

It seems like a lot of people are having problems . We are at a point now that we are convinced it does not work. We can only get as far as:

Has anyone out there ever got this too work ?

Kind regards,

Adam

Attention Australian Domino Users

2009-11-17T05:35:00.004+11:00

To celebrate the rise and rise of the Australian dollar, between now an the end of the year we are offering Defrag.NSF at $AU 199 including GST. Just enter the coupon Go Aussie when you register

Don't forget there is only a couple of weeks left to get your entries into our Defrag competition. Good Luck.

Is Notes 8.51 the World's most modern ICE client ?

2009-11-10T21:50:00.003+11:00

Now that the GFC is lifting, a number of organisations I know are undertaking "Modernisation Projects".

This typically involves looking a Windows 7, and more often than not, installing Exchange and Outlook.

But it got me thinking, Is Outlook 2007 really more modern than Notes 8.5.1 ? Is it? I think not. There might be a couple of things it does better, but overall, and above all, it remains an email client. Notes is much, much more.

So "Is Notes 8.51 the World's most modern ICE client" ? If it is, maybe it should be marketed that way.

Centrelink finally retires Smartsuite

2009-11-07T17:05:00.002+11:00

For the last 15 odd years,Centrelink (the Australian Commonwealth Services Delivery Agency) has been using Lotus Smartsuite. 26,000 copies install across the organisation.

This month they change over to Microsoft Office 2007.

I think the fact that the product has been able to survive this long in the organisation show what an amazing job the software engineers and programmers did when they developed it. The product hasn't changed for ten years, yet the features it offers and the integration it has with Notes are still very advanced. To those developers, I salute you!

When Al Zollar put a sword through Smartsuite's development 10 years ago he canned an awesome product and handed the market to Microsoft for a large number of years.

In Janurary of this year Centrelink deployed Notes 8.02 with the Symphony productivity editors, so I guess Centrelink staff will soon have two World Class office productivity solutions to create documents and develop presentations. Lucky them..

RIP Smartsuite.

Since when did a 100% improvement become an average result, in anything?

2009-11-04T15:27:00.009+11:00

There is only a month to go in our Defrag.NSF competition and some of the results people have sent in are astounding.

Results vary, let say over 100% is rather common. Who wouldn't want a 100% improvement in read times on their server ?

So what could Defrag.NSF do for your?

Go to our website, register for a trial, and send in your entry.

Good luck.

Nobody knows - why there is no Lotus 'air cover' in Australia

2009-10-29T08:41:00.002+11:00

I have just returned from visiting sites in Victoria, Australia. This state has some of the largest Notes Domino customers in the country, and every single customer raised the same concern - lack of marketing and mind share was making life tough.

This has been raised time and time and time again. But still nothing. Yet nobody knows why.

8.5.1 is absolutely awesome. By any definition it must be the most advanced and modern ICE client in the market, Outlook is a toy in comparison, yet without the mindshare people will move away.

IBM we need your help now, air cover is pointless once the troops are dead.

How do you stop folders from being deleted in your archive?

2009-10-26T11:14:00.002+11:00

This is a niggling bug that is driving one of our customers crazy. They archive mail to a server based archive and this mail is in folders (several levels deep). Works well. But when they delete an empty folder from their mail file, the folder in the archive is also deleted. How do you 'turn this feature off'. We have tried changing the replication settings on the archive to not replicate deletions but that does not work.

Anyone seen any "Lotus Knows" campaign activity in Aust or NZ ?

2009-10-14T16:39:00.002+11:00

I've been looking but haven't spotted anything yet. If anyone has seen something I'd love to know.

Cheers

8.5.1 on Snow leopard has a couple of issues

2009-10-13T13:08:00.003+11:00

In-line spell check doesn't work, and there are a number of icons/graphics missing. Otherwise a very nice release.

eg.

A 224% improvement in read times - Can you beat it ?

2009-10-02T15:52:00.004+10:00

When we put together the screen shots for our Defrag competition, we just grabbed one of the logs off one of our servers. It showed a 224% improvement in read time.

I assure you this is nothing special and I would be amazed if you don't do better. Our competition entries to date far exceed this - so what is stopping you?

You could win an iPod nano (actually you could win two) and you will absolutely have a faster server, and way faster backup times. It's a Win-Win situation.

For all the details, check out our web site at www.preemptive.com.au

Win a nano and get a performance boost at the same time !

2009-09-29T12:08:00.004+10:00

It's that time again!

The Annual Defrag.NSF Competition with an iPod nano and 12 month Defrag.NSF license as First Prize and all entries eligible for a second iPod nano Prize Draw.

Last year we were able to track down the World's "Most Fragmented" Domino Server, this year, in the tradition of the "Greatest Loser", we are looking for the Domino Server that has lost the most flab. We are asking for a screen-shot of the Performance Logs view of Defrag.NSF showing the database on your server with the greatest percentage improvement.

The rules are simple, send us the screen-shot and the entrant with the highest recorded percentage in the "Improvement" column will receive a free 12 month Defrag.NSF license for that server so that all that lost flab can be kept at bay, and the new level of performance can be maintained, PLUS, the winner will also receive a new iPod nano, just in time for Christmas!

To enter, simply register for a Defrag.NSF trial license today, run Defrag.NSF (Version 1.37) on your server and submit your entry.

Full Competition and Entry details here!

IBM Lotus Technical University 2009 - Melbourne Australia

2009-06-24T21:23:00.002+10:00

August 25 - 27, 2009

Melbourne

$3300

All the details are here

Does adsync work on Windows 2008?

2009-06-09T20:30:00.003+10:00

We have tried and I know others have, but I can't find any reference that states if it should work or how to actually get it to work.

When we try to register the nadsync.dll on Windows 2008 we get the following errors:

1. 'Lotus ADSync: Access is denied' followed by
2. 'RegSvr32: DllRegisterServer in nadsync.dll succeeded'

Does it work ? Should it work ? Anyone know ?

Centrelink completes its Notes/Domino 8.X roll-out to 26,000+ desktops.

2009-06-01T21:45:00.001+10:00

I just wanted to share some great news, Centrelink (Australia's Commonwealth Services Delivery Agency) has completed its roll-out of Notes 8.02 to all staff and has recently upgraded Domino to version 8.5.

All staff have access to the Notes 8.02 standard client with all its features (integrated chat, Symphony, etc) along with Domino Web Mail. It's been a fantastic process to watch and be part of.

Upgrading 26,000+ users and 450+ servers was never going to be easy, but I think all those involved did an awesome job.

Lotus Protector - it just works !!!

2009-05-12T18:18:00.001+10:00

Recently I got absolutely sick of SPAM entering our organisation and decided to swap out our existing solution for Lotus Protector. To my absolute delight it was so simple. We just downloaded the VM, placed it on our ESX server, performed a quick configuration, redirected mail to the VM and bingo, we were done.

It has been working incredibly well and I'll be recommending it to anyone who is willing to listen.

Good job IBM.

Windows Terminal Server and Notes 8.5 Basic ?

2009-05-08T13:30:00.002+10:00

Hi,

One of our customers has gone down the WTS road and we've been asked about Notes 8.5 running on it. All the IBM documentation says not to do it, but I wonder if Notes 8.5 Basic will work. Has anyone tried it ? and can you offer some tips ?

I would hate to lose this account to Outlook because of this issue.

Notes 8.5 Marketing - has anyone seen any ?

2009-04-09T12:02:00.002+10:00

You know, posters, ads, banners, on the side of a bus, in an airport.

At Lotusphere we were told there would be increased marketing of the Lotus brand, but I'm struggling to find it and certainly haven't seen it, has anyone else ?

Domino 8.5 Exam and CertFX

2009-03-28T08:05:00.001+11:00

Has anyone out there used the CertFX practice test to prepare for the 8.5 Admin update ? If so how did you find it ?