The pros and cons of Legato License Manager

Bundled with NetWorker for some time now has been the peripheral product, Legato License Manager (LLM). (Now normally just referred to as “License Manager”.)

If you’ve never used LLM, you may wonder what use it serves. But to do that, we first need to look at the control zone, that region of space that encompasses one or more NetWorker datazones. This might resemble the following:

Both the NetWorker Management Console (NMC) server (typically the “gst” processes) and LLM reside in the control zone – that is, they exist to service multiple datazones. However, in the same way that many sites end up running the datazone and control zone (via NMC/GST) on the same backup server, there’s nothing preventing you from using LLM to manage licenses separately to the core NetWorker services.

Making this transition is relatively straight forward, and I’ll save doing an article on that aspect unless people would like to see one – instead, I’d like to discuss the pros and cons of having licenses outside of NetWorker but still referenced by a NetWorker server.

Advantages of LLM

It’s best to first understand what LLM brings to you. I’ll use the following keys beside the advantages to let you know where they’re applicable:

• (D+) – Useful for multiple datazones.
• (1D) – Useful for single datazones.
• (M) – Marketing advantage; touted as a bonus by EMC/Legato, rarely if ever used.

So, some of the advantages are:

• (D+) Licenses may be purchased and installed in a central location.
• (D+) Licenses may be reallocated between NetWorker servers as resource requirements/capabilities change (i.e., released from one server, and snapped up by another).
• (1D/D+) License authorisation codes are tied to the LLM server, not the NetWorker server. Therefore if you’ve got an environment where you’re planning on doing some NetWorker server migrations, you can move your licenses to LLM and not have to repeatedly do host transfers.
• (M) You can buy “bulk” licenses. (E.g., 10 x 5 Client Connection Licenses). This advantage seems to be minimising the number of licenses you need to enter. While this sounds cute in theory, I think it actually adds a layer to license complexity.
• (1D/D+) Licenses reported in NMC show the exact features they are being used for – e.g., instead of showing “NetWorker Server, Network Edition/125” to indicate that the server is licensed for 125 clients, you might see “NetWorker Server, Network Edition/93” to indicate that currently there are 93 client licenses being used.
• (M) LLM is where the full version of NMC is licensed, so you only have to learn one licensing administration system.

Disadvantages of LLM

The advantages of LLM don’t come without some disadvantages too. These are:

• Not all licenses work all the time with LLM. For example, historically there has been ongoing issues with creating dedicated storage nodes in an environment using LLM. Typically it’s been necessary to add both a dedicated and a full storage node license, create the dedicated storage node devices, then delete the full storage node license. (Messy.)
• When working outside of NMC, the command line access to LLM licenses (via lgtolic) is even more esoteric and preposterous to use than nsrcap and nsradmin.

Should you use LLM?

It depends on your site. If you’re a fairly small environment, I can’t see any purpose for switching from NetWorker licensing to LLM; however, if your site has a larger number of clients and is reasonably dynamic in client allocations, LLM may give you that extra easy oversight to justify switching to it even if you’ve only got one datazone in your environment.

Alternatively, if you’re planning to transition your backup server a few times over a shorter period (e.g., migrating from current old hardware to interim hardware to full new hardware), then moving licenses out of NetWorker and into LLM may save you the hassle of getting them re-authorised at each step.

Should your LLM server be considered “production”?

If you’re using LLM, the obvious question that some will ask is – can this just be plonked on any old desktop PC? The answer is no. Well, to qualify that answer a bit – nothing prevents you from doing this except common sense. By all means have this running as a virtual machine somewhere, but it should still be considered a production machine in the same way that a backup server is a production machine: it’s part of support production rather than primary production, but it’s still production.

Enhancing NetWorker Security: A theoretical architecture

It’s fair to say that no one backup product can be all things to all people. More generally, it’s fair to say that no product can be all things to all people.

Security has had a somewhat interesting past in NetWorker; much of the attention to security for a lot of the time has been to with (a) defining administrators, (b) ensuring clients are who they say they are and (c) providing access controls for directed recoveries.

There’s a bunch of areas though that have remained somewhat lacking in NetWorker for security. Not 100% lacking, just not complete. For instance, user accounts that are accessed for the purposes of module backup and recovery frequently need higher levels of authority than standard users. Equally so, some sites want their <X> admins to be able to control as much as possible of the <X> backups, but not to be able to have any administrator privileges over the <Y> backups. I’d like to propose an idea that, if implemented, would both improve security and make NetWorker more flexible.

The change would be to allow the definition of administrator zones. An “administrator zone” would be a subset of a datazone. It would consist of:

1. User groups:
   • A nominated “administrator” user group.
   • A nominated “user” user group.
   • Any other number of nominated groups with intermediate privileges.
2. A collection of the following:
   • Clients
   • Groups
   • Pools
   • Devices
   • Schedules
   • Policies
   • Directives
   • etc

These obviously would still be accessible in the global datazone for anyone who is a datazone administrator. Conceptually, this would look like the following:

The first thing this should point out to you is that administrator zones could, if desired, overlap. For instance, in the above diagram we have:

1. Minor overlap between Windows and Unix admin zones (e.g., they might both have administrative rights over tape libraries).
2. Overlap between Unix and Oracle admin zones.
3. Overlap between Windows and Oracle admin zones.
4. Overlap between Windows and Exchange admin zones.
5. Overlap between Windows and MSSQL admin zones.

Notably though, the DMZ Admin zone indicates that you can have some zones that have no overlap/commonality with other zones.

There’d need to be a few rules established in order to make this work. These would be:

1. Only the global datazone can support “<x>@*” user or group definitions in a user group.
2. If there is overlap between two zones, then the user will inherit the rights of the highest authority they belong to. I.e., if a user is editing a shared feature between the Windows and Unix admin zones, and is declared an admin in the Unix zone, but only an end-user in the Windows zone, then the user will edit that shared feature with the rights of an admin.
3. Similarly to the above, if there’s overlap between privileges at the global datazone level and a local administrator zone, the highest privileges will “win” for the local resource.
4. Resources can only be created and deleted by someone with data zone administrator privileges.
5. Updates for resources that are shared between multiple administrator zones need to be “approved” by an administrator from each administrator zone that overlaps or a datazone administrator.

Would this be perfect? Not entirely – for instance, it would still require a datazone administrator to create the resources that are then allocated to an administrator zone for control. However, this would prevent a situation occurring where an unprivileged user with “create” options could go ahead and create resources they wouldn’t have authority over. Equally, in an environment that permits overlapping zones, it’s not appropriate for someone from one administrator zone to delete a resource shared by multiple administrator zones. Thus, for safety’s sake, administrator zones should only concern themselves with updating existing resources.

How would the approval process work for edits of resources that are shared by overlapping zones? To start with, the resource that has been updated would continue to function “as is”, and a “copy” would be created (think of it as a temporary resource), with a notification used to trigger a message to the datazone administrators and the other, overlapping administrators. Once the appropriate approval has been done (e.g., an “edit” process in the temporary resource), then the original resource would be overwritten with the temporary resource, and the temporary resource removed.

So what sort of extra resources would we need to establish this? Well, we’ve already got user groups, which is a starting point. The next step is to define an “admin zone” resource, which has fields for:

1. Administrator user group.
2. Standard user group.
3. “Other” user groups.
4. Clients
5. Groups
6. Pools
7. Schedules
8. Policies
9. Directives
10. Probes
11. Lockboxes
12. Notifications
13. Labels
14. Staging Policies
15. Devices
16. Autochangers
17. etc.

In fact, pretty much every resource except for the server resource itself, and licenses, should be eligible for inclusion into a localised admin group. In it’s most basic, you might expect to see the following:

nsradmin> print type: NSR admin zone; name: Oracle
               type: NSR admin zone;
               name: Oracle;
     administrators: Oracle Admins;
              users: Oracle All Users;
  other user groups: ;
            clients: delphi, pythia;
             groups: Daily Oracle FS, Monthly Oracle FS,
                     Daily Oracle DB, Monthly Oracle DB;
              pools: ;
          schedules: Daily Oracle, Monthly Oracle;
           policies: Oracle Daily, Oracle Monthly;
         directives: pythia exclude oracle, delphi exclude oracle;
...etc...

To date, NetWorker’s administration focus has been far more global. If you’re an administrator, you can do anything to any resource. If you’re a user, you can’t do much with any resource. If you’ve been given a subset of privileges, you can use those privileges against all resources touched by those privileges.

An architecture that worked along these lines would allow for much more flexibility in terms of partial administrative privileges in NetWorker – zones of resources and local administrators for those resources would allow for more granular control of configuration and backup functionality, while still keeping NetWorker configuration maintained at the central server.

Your datazone is only as secure as your NetWorker server

A topic I discuss in my book that’s worth touching on here is that of datazone security.

Backup is one of those enterprise components that touches on a vast amount of infrastructure; so much so that it’s usually one of those most broadest reaching pieces of software within an environment. As such, the temptation is always there to make it “as easy as possible” to configure. Unfortunately this sometimes leads to making it too easy to configure. By too easy, I mean insecure.

Regardless of the “hassle” that it creates, a backup server must be highly secured. Or to be perhaps even blunter – the entire security of everything backed up by your backup server depends on the security of your backup server. Having an insecure NetWorker server, on the other hand, is like handing over the keys to your datacentre, as well as having the administrator/root password for every server stuck to each machine.

Thinking of it that way, do you really want the administrator list on your backup server to include say, any of the following?

• *@*
• *@<host>
• <user>*@

If your answer is yes, then you’re wrong*.

However, datazone security isn’t only about the administrator list (though that forms an important part). At bare minimum, your datazone should have the following security requirements:

1. No wild-cards shall be permitted in administrator user list definitions (server, NMC).
2. No client shall have an empty servers file (client).
3. No wild-cards shall be permitted in remote access user list definitions (client resources).

Note: With the advent of lockboxes in version 7.5, security options increase – it’s possible, for instance, to have passwords for application modules stored in such a way that only the application module for the designated host can retrieve the password.

—
* I do make allowance for some extreme recovery issues that have temporarily required users to enter wild-card administrators temporarily where it was not possible to wait for a bug fix.

Zeroth tier: the backup director

(OK, I just made that term up, there is within the NetWorker framework, no reference ever to a “zeroth” tier. That doesn’t preclude me from using the term though.)

The classic 3-tier architecture of NetWorker is:

• Backup Server
• 1 or more storage nodes (1 of which is the backup server)
• Clients

In a standard environment, as it grows, you typically see a situation where clients are hived off to storage nodes, such that the backup server handles only a portion of the backups, with the remainder going to storage nodes.

One thing that’s not always considered is what I’d call the ability to configure the NetWorker server in a zeroth tier; that is, acting only as backup director, and not responsible for the data storage or retrieval of any client.

Is this a new tier? Well, technically no, but it’s a configuration that a lot of companies, even bigger companies, seem reluctant to engage in. It seems for the most part that this is due to the perception that by elevating the backup server to a directorial role only, the machine is ‘wasted’ or the solution is ‘costly’. Unfortunately this means many organisations that could really, really find benefit in having a backup server in this zeroth tier continue to limp along with solutions that suffer random, sporadic, periodic failures that cannot be accounted for, or require periodic restart of services just to “reset” everything, etc.

Now, the backup server still has to have at least one backup device attached to it – the design of NetWorker requires the server itself to write out its media database and resource database. There’s a good reason for this, in fact – if you allow such bootstrap critical data to be written solely to a remote device (i.e., a storage node device), you create too many dependencies and setup tasks in a disaster recovery scenario.

However, if you’re at the point where you need a NetWorker server in the zeroth tier, you should be able to find the budget to allocate at least one device to the NetWorker server. (E.g., a bit of dynamic drive sharing, or dedicated VTL drives, etc., would be one option.) Preferably of course that would be two devices so that cloning could be handled device<->device, rather than across the network to a storage node, but I don’t want to focus too much on the device requirements of a directorial backup server.

There’s actually a surprising amount of work that goes into just directing a backup. This covers such activities as:

• Checking to see what other backups at any point need to be run (e.g., multiple groups)
• Enumerating what clients need to be backed up in any group
• Communicating with each client
• Receiving index data from each client
• Coordinating device access
• Updating media records
• Updating jobs records
• Updating configuration database records
• etc.

If the grand scheme of things where you don’t have “a lot” of clients, this doesn’t represent a substantial overhead. What we have to consider though is the two different types of communication going on – data, and meta-data. Everything in the above list is meta-data related; none of it is actually the backup data itself.

So add to the above list the data streams that have one purpose in a normal backup environment – to saturate network links to maximise throughput to backup devices.

Evaluating these two types of communication – meta-data streams and data streams, there’s one very obvious conclusion: they aren’t mutually satisfying. That is, the data stream is by necessity going to be as greedy with bandwidth as it can be, and just as equally, the meta-data stream must have the bandwidth it requires or else failures start to happen.

So, as an environment grows (or as NetWorker is deployed into a very large environment), the solution should be equally as logical – if it gets to the point where the backup server can’t facilitate meta-data bandwidth and regular data bandwidth, there’s only communications stream that can be cut from its workload – the data stream.

I’m not suggesting that every NetWorker datazone needs to be configured this way; many small datazones operate perfectly with no storage nodes at all (other than the backup server itself); others operate perfectly well with one or more storage nodes deployed and the backup server operating as a storage node. However, if the environment grows to the point where the backup server can be kept fully occupied by directing the backups, then cut the cord and let it be the director.