Basics – jbverify

Now, some might say that I’m not the smartest card in the deck for what I’m about to note, but sometimes I don’t notice new commands when they appear in NetWorker, particularly if I skimmed through release notes.

I was pleasantly surprised today to find that a new command had slipped in at some point called “jbverify”. I can immediately see it however entering my stable of must-use commands, particularly in a support environment.

To quote the man page, jbverify:

[V]erifies the devices defined in the NetWorker database, making sure that each one of them is configured properly by checking them  for accessibility  and  usability.

This is the sort of diagnostic tool that support people live for, and sites suddenly experiencing strange jukebox issues should think of as a matter of course.

When run on my lab server this afternoon, I got the following badly formatted but still very useful output:

# jbverify
14866:jbverify:
 Jbverify is running on host nox, Linux 2.6.18-128.1.6.el5

14912:jbverify:
 Processing stand-alone devices...

14913:jbverify:
 Processing /d/nsr/02/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/02/_AF_readonly

14913:jbverify:
 Processing /d/nsr/idata/backup/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/idata/backup/_AF_readonly

14913:jbverify:
 Processing /d/nsr/01/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/01/_AF_readonly

14913:jbverify:
 Processing /d/nsr/idata/backup
14915:jbverify:
 Finished processing /d/nsr/idata/backup

14913:jbverify:
 Processing /d/nsr/idata/clone/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/idata/clone/_AF_readonly

14913:jbverify:
 Processing /d/nsr/01
14915:jbverify:
 Finished processing /d/nsr/01

14913:jbverify:
 Processing /d/nsr/03
14915:jbverify:
 Finished processing /d/nsr/03

14913:jbverify:
 Processing /d/nsr/03/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/03/_AF_readonly

14913:jbverify:
 Processing /d/nsr/02
14915:jbverify:
 Finished processing /d/nsr/02

14913:jbverify:
 Processing /d/nsr/idata/clone
14915:jbverify:
 Finished processing /d/nsr/idata/clone

14917:jbverify:
 Finished processing stand-alone devices.

14918:jbverify:
 Processing jukebox devices...

14920:jbverify:
 Processing jukebox LTO1_LIB:

14733:jbverify:
 Testing drive 1 (/dev/nst0) of JB LTO1_LIB

14927:jbverify: 

 Jukebox LTO1_LIB on nox successfully processed.

14929:jbverify: 

 Finished processing jukebox devices.

**********************************************************************

         Summary report of jbverify
         ======= ====== == ========

Hostname   Device Handle    Blocksize  Jukebox  Drv No. Status
--------   -------------    ---------  -------  ------- ------
nox        /d/nsr/02/_AF_readonly 131072 N/A    N/A     Pass
nox        /d/nsr/idata/backup/_AF_readonly 131072 N/A N/A Pass
nox        /d/nsr/01/_AF_readonly 131072 N/A    N/A     Pass
nox        /d/nsr/idata/backup 131072  N/A      N/A     Pass
nox        /d/nsr/idata/clone/_AF_readonly 131072 N/A N/A Pass
nox        /d/nsr/01        131072     N/A      N/A     Pass
nox        /d/nsr/03        131072     N/A      N/A     Pass
nox        /d/nsr/03/_AF_readonly 131072 N/A    N/A     Pass
nox        /d/nsr/02        131072     N/A      N/A     Pass
nox        /d/nsr/idata/clone 131072   N/A      N/A     Pass
nox        /dev/nst0        65536      LTO1_LIB 1       Pass

**********************************************************************

If you’ve come from NetBackup, the nature of this program is somewhat reminsicent of the robtest utility. I don’t claim EMC are special for having introduced this tool, but I do applaud that it’s there (and lament that I didn’t notice it sooner).

(One thing to note: after running jbverify, make sure you reset your jukebox.)

Basics – Using datazone encryption with NetWorker

I’m not fond of software encryption (or compression, for that matter). Particularly in a 24×7 enterprise environment, clients (i.e., production servers) have better things to be doing than doing on-the-fly software encryption or compression. In these environments, hardware encryption routers should be the product of choice for achieving totally secure backups. Such devices also have advantages in terms of key management – much more flexible, scalable and appropriate for role based data access.

That being said, in smaller environments, or environments where servers are relatively idle overnight, NetWorker’s datazone encryption can be sufficient to achieve a reasonable modicum of backup protection with minimum effort – and most importantly, cost.

To get started using NetWorker datazone encryption, you first need to assign a pass phrase. This is done in the NetWorker server properties (typically accessed within NMC):

With the pass phrase in place, you can then configure directives within NetWorker to make use of AES 256 bit encryption. However! As soon as you turn encryption on, you lose all potential for hardware based compression for your media. Why? Quite simply, compression is about finding patterns in data and reducing all the matching patterns to a single reference point; however, encryption is all about eliminating patterns, making the data appear completely random.

Thus, if you want to still get some measure of compression, you should, when using this method, employ software based compression in your directive as well.

Thus, a base directive might look like the following:

<< / >>
+compressasm: .
+aes: .

This will apply compression first to all files encountered, then once the file has been compressed, it will be encrypted. A side benefit of this is that by compressing first, you reduce the amount of data to be encrypted*.

So long as the datazone pass phrase is stored in the server, encryption will occur, and no password will be required to recover the data. Remember, this style of encryption, using a single pass-phrase, isn’t about being able to restrict whom within the datazone can recover the data, but instead it’s about keeping the data stored on-tape (which is potentially off-site, or otherwise at higher risk of theft), from being recovered.

[Edit, 2009-08-15]

It’s been pointed out to me that you can’t compress + encrypt at the client side. Indeed, I’ve now found the part in the administration guide that explicitly says this. What is extremely disappointing about this is that NetWorker actually doesn’t warn you that it’s not going to compress + encrypt! To me, that’s a security issue.

So, for the examples above, forget about enabling client side compression as well as encryption – you can have one or the other, but not both.

—
* In the same way that ice-cream that’s 99% fat free, but 87% sugar is a “benefit”.

Basics – Remote Control

I learned this technique several years ago before NetWorker supported running an nsrjb command from the backup server to manipulate a jukebox on a storage node. (Previously it had required that actual nsrjb commands, when run from the command line, be run from the owner of the jukebox.)

While you don’t need it any more to control remote jukeboxes, it can still be useful to know how to do remote control operations in NetWorker – particularly if you’re say, debugging backups on a client but you don’t have console access to that client.

Note that this only applies to commands that obey the following restrictions:

• Name starts with “nsr” or “save”
• Resides in the same directory as the “save” command on the client.

Thus, this isn’t about remote hijacking of a NetWorker client. Such commands can only be executed by an authorised administrator on the backup server as specified in the nsr/servers file. Before you (quite rightly) point out that it would mean that a valid NetWorker administrator could in fact hijack a client by say, doing a directed recovery out to that client of an appropriately named file, they can do that already – that’s part of the trust relationship of the NetWorker administrator anyway.

So, with all those caveats out of the way, it’s remarkably simple. Going from a Unix host, you do the following:

# export RUSER=<user>
# export RCMD=<cmd>
# nsrexec -c <client>

Where:

• user will typically be “root” or “administrator”, depending on whether your backup server is Unix or Windows.
• cmd will be a NetWorker command you want executed.
• clientName is the name of the client the command is to be executed from.

For instance, say I’ve got a NetWorker server called “nox”, and a client called “asgard” that I don’t have administrative login to, but I want to simulate a backup without firing up a savegroup. To do so, I could do the following:

# export RUSER=root
# export RCMD="save -e tomorrow -b Default -LL -q -s nox /tmp"
# nsrexec -c asgard
save: /tmp  36 MB 00:00:05     38 files
completed savetime=1235278395

Admittedly this is not a technique you should need to know often, but it’s useful to know about.

—
* You do populate your nsr/servers file, don’t you? If you don’t, go do it NOW. I mean it, stop what you’re doing, go and fix up the nsr/servers file on every client in your environment!

Basics – Changing saveset browse/retention times

Hi!

The text of this article has been moved, and can now be found at its permanent home, the NetWorker Information Hub. You can read it here.

Basics – Stopping and starting NetWorker from the Windows command line

Hi!

The text for this article has moved, and can should now be accessed from the NetWorker Information Hub. You can read it here.

Basics – Matching savesets with files on disk backup units

Generally speaking, you don’t want to be mucking around with the contents of your disk backup units except under extreme circumstances. In fact, I really recommend that you don’t do so unless you 100% know what you’re doing.

So this post is all about that 1-5% of the time where you may find it necessary to say, search for a saveset that’s reported in the media database that you’re having problems accessing from the disk backup unit.

It’s actually trivially easy, once you know how.

You may be familiar with the following style of query:

# mminfo -q "savetime>=24 hours ago" -r "volume,client,level,sumsize,savetime(22)"

The (22) at the end of the savetime report parameter tells mminfo to allow 22 characters for the reporting of the savetime. The benefit of this is that you get not only the savetime date, but also the time as well.

NetWorker actually allows you to put that (number) postfix onto any field that it can output in mminfo. This can output additional information, such as the above, or give more room to output longer fields, or even limit the size of fieldnames when you don’t need too much information. (E.g., if the first 4 characters of all client names can uniquely identify the client, you might limit the client to 4 characters in an mminfo report.)

Now, where we’re heading with this, is that the sorts of filenames used for the savesets written to disk backup units are not some random collection of strings – they’re actually the long saveset ID.

Consider then a filename of:

/d/nsr/02/63/05/cd3a182f-00000006-7b7801de-497801de-01871a00-3d2a4f4b

This isn’t just a random filename, it’s the saveset ID, but just in a format you may not used to.

To get the long saveset ID in mminfo output, we use the (number) postfix on the ssid field. This would be as follows:

# mminfo -q "ssid=071462366" -r "ssid(53)"
cd3a182f-00000006-7b7801de-497801de-01871a00-3d2a4f4b

With that information in hand, you can then search for a file with the same name as the long saveset ID on disk.

You can also do a reverse lookup. Say for instance, you know there’s an issue with a particular saveset file on a disk backup unit. To find out what the actual saveset ID is for this saveset, you can run the counter-query:

mminfo -q "ssid=cd3a182f-00000006-7b7801de-497801de-01871a00-3d2a4f4b" -r ssid

So, there you go – very easy!