Sonic Tech Note

How to Clone a Sonic Directory Service

Dave Hentchel

October 23, 2007

Introduction

Many SonicMQ customers have nearly identical configurations for multiple SonicMQ installations, particularly when going from development, through QA, to the deployed brokers.  In some cases low-level configuration changes may be done in one environment, but if these are not replicated in the other environments you lose the advantage of building and testing with the 'true' configuration.  Therefore, there is an advantage to being able to quickly and cleanly transfer the complete, identical configuartion across environments to ensure they are true replicates.  Of course, it will normally be necessary to modify the host address component in connection URL's and possibly directory file paths, if filesystems are not identically configured across machines.

The Sonic supported technique for transferring configurations across environments is the Sonic Deployment Manager.  It leverages the loosely-coupled nature of Sonic technology and embeds calls to installation programs, import/export api's and mapping tools to provide maximum flexibility in automated provisioning of remote systems. The main disadvantages of SDM are the costs to purchase and the ramp-up time to learn how to effectively use it. Advantages of SDM include:
This paper describes a brute force cloning technique that has been used in some circumstances to create a copy of a given configuration in a new environment. Basically it involves copying the Directory Service data files, then tailoring them to reflect the new host names and IP addresses of the target environment.  Some things to note about this approach:
Because this approach is not tested in the Sonic product QA suite, you should use appropriate caution.  For example, we recommend, if at all possible, that you maintain your Production system using supported tools and use this cloning technique to build replicated QA and Test systems, rather than going the other way around.  If you do use 'upstream' cloning, make certain you carefully test your procedures, keeping in mind that new releases may add new connection definitions that will require additional tailoring.

Prerequisites

Here are the checks you should perform prior to doing a cloning operation:
  1. The source and target systems must be nearly identical
  2. If at all possible, use the same absolute directory path location for each install, and make sure all directories used in the config exist on both machines.  Important directory paths to check include:
  3. You can use relative file paths, but this increases the openings for operational errors.
  4. ESB Container classpath should contain only sonicfs:// style references, i.e. all needed files are in the DS itself.
  5. Similarly, Event Monitor log4j config files should reside only in sonicfs://
  6. Make sure both source and target Domains are at identical release/patch levels.  At minimum, make certain that all lib and Archives directories are identical across environments.  Container bootstrap and startup files are less critical.
  7. If cloning from an Integration Workbench domain into a regular install, all files in sonicfs://workspace will be lost, because they are actually stored on the developer's filesystem and not in the sonicfs://.  You should make sure that no deployed ESB process or service uses workspace files, or you will have to independently export and import that directory.
  8. Similarly, the dev_XXXX containers will be included in any DS cloned from a Workbench environment; they can be simply ignored on the production side or you can manually delete them.  It is not advisable to run them in production unchanged, because they are started in Test mode, which would allow insecure access via the debugger.
  9. Back up the target system's Directory Service (i.e. the subdirectory of SonicMQ with the same name as the Domain) so you can recover quickly if the process gets interrupted or hits a problem.
  10. If there are any changes in the number of containers, location of files, etc, synchronize those file directories for the two systems manually.
  11. Shut down all Sonic containers
  12. Delete all container cache files (directory names like DomainName.ContainerName.cache)
Note that there are other operational tricks you can perform to further simplify this process.  For example, your cluster and DNS software may allow you to use identical host names in different environments (perhaps located in separate network sub-domains, or maybe using different DNS servers)

Cloning steps

These steps are based on general field experience.  If you discover additional steps or caveats that apply, please let us know.
  1. Copy the Domain from the source system to the target
  2. Open the DS file with Sonic Management Console
  3. Modify the following URLs with the target's host name or ip addresses:
  4. Check file system paths to ensure they are correct for the local system. Specifically:
  5. In a few specific circumstances, such as turning Security on or off, the broker data storage had to be recreated on the source system; in these cases you will also have to recreate storage on the target system.
  6. If the Control Numbers for product licenses differ between the systems, this should be corrected; this is especially true if an evaluation license is in use on the source system, as it will cause the target system to become unavailable when the trial license period expires.
  7. You should run a base series of tests prior to bringing the target system online, carefully checking the container log files for errors. You will then need to ensure that all containers are restarted or reloaded, to refresh their DS image. Your recourse, if anything fails, is to fix the incorrect reference manually or to restore the backup you took prior to the cloning procedure. This might also be a good time to take a second look at the Sonic Deployment Manager.