Fingers Crossed for some New Years Resolutions from EMC

Over at Storagebod’s Blog, Martin Glassborow has been providing a highly interesting and entertaining set of letters to Father Christmas, which I’d highly recommend reading.

So in the spirit of Martin’s postings, I’m going to do a slight variant – here’s a few things that I’ll be keeping my fingers crossed for that EMC will provide us in NetWorker in 2010.

Dear EMC,

Could you please add to your NetWorker New Years Resolution the following items?

• Windows 2008 Service Pack 2 support as soon as possible!
• Improvements for ADV_FILE devices:
  • Savesets can spill over from one disk backup unit to another should the first become full.
  • Better rotation/selection method for devices/volumes.
• Tapes:
• Ability to query the “offsite” mminfo flag.
• Cloning:
  • Inline cloning (simultaneous clone + original generation).
  • Fixing the “validcopies” flag!
• NetWorker Management Console:
  • Manual client backup and recovery operations integrated into the console.
  • Manual module backup and recovery operations integrated into the console.
• Resources/Configuration:
  • Ability to rename clients (and other resources).
  • Syntax for including another directive within a directive.
  • PIDs for nsrmmds stored within the device resource for easy viewing.
• Operations:
  • Client GUI manual backups to support pools other than Default.
  • More granular savesets.

I could think of a bunch of other enhancements, but these are the ones I’m hoping most are on your new years resolution list for NetWorker in 2010. I’m hoping I don’t have to put any of these on a wish list for your 2011 new years resolution list.

Thanks, and have a happy new year!

Preston de Guise.

My Personal NetWorker Wish List

This is not necessarily reflective of my customers, though several of them have expressed desire for at least one or two features in the following list. However, that doesn’t mean there’s features currently not in NetWorker that I feel would significantly enhance it.

(I should point out that even missing these features, I still think it’s superior to other backup products.)

So, here’s my personal wish-list for NetWorker (in no particular order), coming from a continuous use of the product since 1996:

• Enhancements to ADV_FILE backup:
  • The nsrmmd service needs to be able to support some form of proxying, such that savesets which are being written to a disk backup unit which fills can be “moved” to another nsrmmd service for completion on another ADV_FILE unit.
  • When backing up to ADV_FILE units, NetWorker needs to pick next volume to write to by capacity, not age. This would prevent the same ADV_FILE unit(s) filling frequently and stagger writes better across all devices.
• Enhancements to backup, more generally:
  • Simultaneously generate a backup and one or more clones, with the option available as to whether failure of one, a particular number or all constitutes a failure.
  • Support for expiration/browse dates beyond 2038.
• Enhancements to recovery:
  • New recover log that gets automatically populated with details of who recovered what. This should be at the NetWorker server, rather than NMC level.
  • Administrators should be able to optionally allow cross platform directed recoveries. (This will become particularly pertinent as companies complete migration activities from Novell NetWare to Novell Open Enterprise Server. As one platform is NetWare, and the other Linux, recovering old NetWare data to an OES server is not supported.)
  • Preferred recovery pools – currently NetWorker picks savesets to use for recovery based on nsavetime, which can result in disk backup environments where data has been cloned, then staged, in NetWorker requesting recovery from a clone rather than stage volume. While this can be ameliorated by individually setting the “offsite” flag for each volume, it would be preferable to be able to nominate “priorities” for pools, such that NetWorker will recommend volumes from the “highest priority pool” when a recovery is required and no volumes are on-line.

The wish list is by no means complete, but these are the things I tend to chafe against more frequently than others.

Since I don’t want it to be said that I just come up with a wish-list but don’t have any idea of how topics on it would be accomplished, I’ll go focus on what would (theoretically) need to be done to fix up disk backup, accomplish simultaneous (inline) cloning, etc. (Admittedly, this is my “I don’t have access to the NetWorker source code and never have but architecturally I think EMC should do this” train of thought, but that doesn’t invalidate the theoretical architecture.)

The current “tiering” used to get data to a volume needs to be extended by one layer. Currently the (general) process is that the client save process (or other nominated process) sends data to the nominated nsrmmd process for writing directly to a volume.

It’s this model that needs to be changed. Rather than the client sending directly to the nsrmmd process, instead it needs to send to a proxy process – let’s call that the fictional nsrmmpd process. This process would act as a “broker”. It would receive the individual backup processes from each client backup, and determine which nsrmmd would facilitate the backup. The important feature however would be that there would not be a 1-to-1 relationship between nsrmmds and nsrmmpds – rather, there would be a 1-to-1 relationship between client backups and nsrmmpds, and the nsrmmpd would be able to redirect the client data stream to another nsrmmd on an as-needs basis. This would resemble the following:

Proposed nsrmmpd architecture

The advantage of this style of daemon layout would be that client backup processes would not have to re-negotiate with nsrmmd processes, but rather, would be passed-through by the proxy process to whichever nsrmmd process was most suitable at any given time. This would in theory be completely seamless and undetectable to the client.