CERN/RD45/98/09

3 Mai, 1999

DRO_TOOL (Data Replication Option Tool) is a tool for managing the configuration of a federated database. With this tool the database administrator is able to observe, control and manage the fault tolerance, and detect failures in the system as soon as possible.

Objectivity/DB provides tools and programming interfaces to help perform administration tasks. The tasks and tools are similar on all platforms, but without a friendly interface (most of them are command line programs).

DRO_TOOL is a visual tool that offers the same functionality than the command line tools includes with the Objectivity/DB, and some more functions included in the Java bindings. The tool has been implemented in Java to avoid the multiplatform problems. We have successfully tested DRO_TOOL in Windows NT and Solaris (the only platforms that have the JDK 1.1.7 or later 1.1.x).

The tool has been developed using the JDK 1.1.7, and uses the JFC Swing package 1.1beta2. It also uses the Voyager package from Objectspace (see agents section on the manual).

To get a copy of the program, please consult the web page located at: http://wwwinfo.cern.ch/asd/rd45/tools/dro_tool.html

This report does not want to explain how to administrate a Objectivity/DB federated database. Report to the "Objectivity/DB Administration" and "Using Objectivity/FTO and Objectivity/DRO" books for more information.

This tool should be used by database administrators only. It will help them in their task of administrating a federated database. You cannot use DRO_TOOL as an object browser, but only for administrative purposes.

We assume that the user of this tool and guide has knowledge about administrative tasks with Objectivity/DB. This guide explains the basic concepts, but not in details.

The application main window is divided in two parts. On the left side, there is a tree with the autonomous partition and databases. The root of the tree is the federated database, and as children, it has the autonomous partitions. Every autonomous partition has as children the databases they control. On the right side, there is a text area information panel, where the results of some operation are displayed.

To run the DRO_TOOL application, you need the JDK 1.1.7 beta 2 or later (available at http://java.sun.com/products/jdk/). DRO_TOOL uses the JFC Swing package 1.1beta2 , and Objectivity/DB 5.1 with the Java bindings. For the Data Replication Option part, you also need the Objectivity/DRO component installed in your system.

An Objectivity/DB system consists of multiple processes and files that can be distributed across multiple host machines on a network. The figure above shows a possible configuration of a Federated Database, with hosts distributed on WAN and LAN networks.

• Federated Database: is the higher level in the Objectivity/DB logical storage hierarchy. Physically, each federated database exists as a federated database file, which contains the system database with information about the schemas and a catalog of the databases that make up the federation.
• Database files: is the second highest level in the Objectivity/DB logical storage hierarchy and is where the application’s persistent data is stored. A database consists of one or more logical structures called ‘containers’, which in turn contain the persistent data (basic objects).

• Autonomous Partition: is a mechanism for dividing a federated database into independent pieces, so that each autonomous partition is self-sufficient in case a network or system failure occurs in another partition. Each autonomous partition controls the access to the databases and containers that belong to it. For every autonomous partition, you have a boot file and an autonomous partition system DB file.
• Replicated Databases: autonomous partitions are the foundation for using the Objectivity/DB Data Replication Option to create and manage multiple copies of a database. Database applications automatically access to the closest image, reducing network traffic. If an image becomes unavailable due to network or system problems, work may continue with an available image.
• Journal files: whenever a transaction starts, it records update information in one or more journal files, which are used to return the federated database to its previously committed state if the transaction is aborted or terminated abnormally.
• Boot file: contains information about the configuration of a federated database or autonomous partition, and used by the application or tool to locate and open a federated database.

A federated database consist of two files: the federated database file, where is stored the schema for the federated database and contains a catalog of member database system names and locations, and the federated database boot file, which contains information about the configuration and is used by the application to locate and open the federated database.

The options offered by DRO_TOOL to work with the federated database are:

To create a new federated database, go to the menu FDB® New to open a dialog where you can introduce the attributes that describes the physical and logical structure of the federated database:

• Federated database Id: the unique positive integer that identifies the federated database to the lock server.
• Federated database file host: host where will reside the FD file.
• Federated database file path: path and file name of the FD file.
• Lock server host: the host system on which the lock server for the entire federated database is running.
• Boot file path: the host system and directory path for the boot file of the federated database.
• Journal directory host and path: the host system and directory path for the journal files.
• Page size: the size (in bytes) of the unit of transfer to and from memory and across the network for the entire database.

DRO_TOOL provides two tools that you can use to get information about a federated database. These tools provide the current values of some federated database attributes.

To display the information of a federated database, select the root node in the tree, and click on ‘Properties’, in the ‘FDB’ menu. This will show information about the federated database name, ID, lock server host, boot file and the number of autonomous partitions and database contained.

Another tool provided by Objectivity/DB to list the values of the federated database attribute is oodumpcatalog. This tool can be called from DRO_TOOL by selecting the ‘Dump Catalog’ option in the ‘Tools’ menu. oodumpcatalog list the full pathnames of all files associated with a federated database.

There are some settings for the access mode to a federated database, locks, cache and AMS usage. This can be established from the ‘Settings’ option in the ‘FDB’ menu. There are four different sets of options:

• Access to an autonomous partition: you can enforce the off-line access to all autonomous partition (see section 5.3 for more information about off-line autonomous partitions).
• Locking: you can specify that you don’t want to use the lock server, enable or disable hot mode for all sessions (enabling hot mode causes Objectivity/DB to delay performing certain overhead operations for frequently accessed objects, until they are no longer being accessed), and the time to wait for a lock hold by any other session.
• Cache: establish the Large Object Memory Limit, the number of initial cache pages and the maximum number of pages.
• AMS: set the federated database to always use the AMS server in all transfers.

An autonomous partition is a mechanism for dividing a federated database into independent pieces, so that each autonomous partition is self-sufficient in case a network or system failure occurs in another partition. Although data physically resides in database files, each autonomous partition controls access to the databases and containers that belong to it. Autonomous partition achieve this because each has all of the system resources necessary to run an Objectivity/DB application autonomously, including:

• A boot file
• A lock server
• Schema information
• A list of all autonomous partitions and their locations
• A list of all databases and in which autonomous partition they are in

Because of these features, failure of one machine only affects access to data within its autonomous partition instead of the entire federated database.

The first autonomous partition in a federated database is created implicitly when you create the federated database. You must create any additional partitions explicitly. Creating an autonomous partition also creates its boot file.

To create a new autonomous partition, the majority of the autonomous partitions must be available. You need to specify all the parameters for the new autonomous partition:

• Name
• Lock server host
• Host where the system DB will be stored
• File path and name for the system DB file
• Boot file host
• Boot file path and file name
• Journal files host
• Journal files directory

You can use this boot file to access to this autonomous partition or the federated database from an application.

When you delete an autonomous partition, you:

• Delete the autonomous partition system database.
• Delete the autonomous partition boot file.
• Relocate all containers controlled by the target autonomous partition to their containing databases.
• Delete database images if multiple images exists.

You cannot delete an autonomous partition if it contains the only image of a database. You also need access to the majority of partitions, including the partition to be deleted.

You may want to prevent access to certain autonomous partitions, even if they are physically available and accessible. To make an autonomous partition unavailable, you can configure it as offline. You can later configure it as online if you need to access it.

To mark an autonomous partition as offline or online, go to the AP menu and select the option ‘Set on-line’ or ‘Set offline’.

When you open the dialog window to select the autonomous partition you want to put off-line, only current on-line autonomous partitions will be displayed. The same thing occurs to put on-line an autonomous partition: only the current off-line autonomous partition will be displayed. An error message dialog will be generated if there are not any autonomous partition to put on-line/off-line.

As for the Federated database, select an autonomous partition in the tree and go to the ‘Properties’ option in the ‘FDB’ menu. This will display information about AP name, id, status, AP system database file, AP boot file, journal files directory and lock server host.

Databases are logically and physically where the persistent objects are stored. Because each database is a separate file, you can use databases to distribute data across the network. Administering a database mainly involves updating its storage characteristics to help you better use the disk and network resources.

Like a federated database, a database has attributes that specify its various physical and logical characteristics. Physically, a database is a file, so you must specify the file host and file path. Logically, a database has an identity within a federated database, so its logical attributes are the containing federated database, the system name, and a database identifier. Also, each database has an attribute for the controlling autonomous partition.

A database can be replicated so different images of the same database belong to different autonomous partitions. Each image has an additional attribute specifying its weight.

To create a new database using DRO_TOOL, select the ‘New’ menu item in the ‘DB’ menu. You need to specify the database name, autonomous partition that will control the access to this database, the name of the host where the file will reside, path and file name and weight. By default, the weight of the database is 1. There are also two parameters affecting the containers in this database: the number of initial pages that will be created when you create the database and the growth factor.

When you create a new database, it is created in the default autonomous partition, and then, it will be moved to the specified partition.

Deleting a database consists of deleting the database file and the catalog in the federated database. You can only delete a database if it has no replicas. If the database has replicas in other partitions you have first to remove them. With this option, you cannot remove database images. You need to specify the autonomous partition that contains the database, and the name of the database.

With this option, you can change the database location. Select the ‘Move’ item in the ‘DB’ menu, and select the database you want to move. Once selected, a new window displays the actual setting (host name, db file path and db file name). This operation updates the catalog and physically relocates the file, but make no changes to the logical database attributes.

Working in a DRO environment, you will need to know if a database or image is available or has enough quorum to perform some operations with a database. To know if a database has enough quorum, select the item ‘Check quorum’ in the ‘DB’ menu, and type the database you want to check.

As for the federated database and autonomous partition, you can retrieve the information of a database by selecting it in the tree, and clicking in the ‘Properties’ item of the ‘FDB’ menu. This will show information about the database name, partition that contains this database or image, ID, weight, database file host and name, and the number of replicas and containers for this database. For every container in the database, it shows the name, container ID, number of pages, the growth factor, and the autonomous partition that controls the access to this container, in case that is not the same partition of the database.

If you have Objectivity/DRO, you can create additional images of a database in other partitions. Each image of a database belongs to a single partition, and each partition can contain at most one image of any single database.

Read operations access only one image of a database, usually the closest one available. If there is an image of the database in the boot partition of the application, the process will use that image. Otherwise, the system reads a single image in a different partition. Update operations are written in parallel to all images.

Once you have a database, to create a new image in another partition, go to the ‘Replica’ menu and click on ‘New’. You must specify the database, its autonomous partition and the new target autonomous partition, as well as the host of the new image and the path and name for the database image file. The system will copy the database to its new location, and update the autonomous partitions DB system. As it needs to modify all the autonomous partition DBs, most of them must be availables to perform this operation.

A database image can be delete using the ‘Delete’ option in the ‘Replica’ menu. You need to specify the autonomous partition and the name of the image, and if you want to delete the image if it is the last replica of the database. This operation will delete the image file from the disk, and update all the autonomous partitions and federated database system DB.

You can change the autonomous partition that controls the access to a database or image. It will only create a new image of the database in the target autonomous partition and deletes it from the current partition, but nothing in the database, containers or objects will change. They will have the same ID, and the database file is not moved.

By default, all the database and images have as quorum 1. It can be modified in order to alter the availability of a database with replicas, when there are replicas non availables.

This parameter is used to calculate if a database has enough quorum to be modified or any other operation that requires to read or write into the database.

In some configurations, the number of images accessible to an application may not be sufficient to constitute a quorum. For these cases, you can assign weights to the images to ensure a quorum can be reached.

To test if a database image is accessible from this process, go to the ‘Replica’ menu and click on the ‘is Accessible’ option. It calculates if the selected image has enough quorum to be read or modified.

There are a set of Objectivity tools that can be launched from DRO_TOOL, and obtain the result of those programs inside the text area of the tool. To find more information about every tool, go to the Objectivity/DB manual.

The tools that can be launched from DRO_TOOL are:

• View Schema: shows the schema of all the classes contained in the federated database
• Dump Catalog: lists all the files in the federated database.
• Check Lock Server: checks if the lock server is running in the system.
• Start Lock Server: if the lock server is not running, starts a new lock server process.
• Kill Lock Server: stops the lock server.
• Lock Server Monitor: lists all the processes and locks currently managed by the lock server.
• Start AMS Server: starts the local AMS server, if not running
• Stop AMS Server: kills the AMS server process.
• Clean Up: lists all the active transactions for the federated database or autonomous partition, and recover the transaction started by local processes.
• List Wait: displays information about transactions waiting for any lockable Objectivity/DB object.
• Space: ??
• Tidy: consolidates a fragmented federated database.

8.1. Statistics

The idea of adding some statistics tests in the program was for obtaining more information about the AMS server throughput and the state of the communication between the servers involved in the federation. As we don’t have enough access to control the low-level communications between the AMS servers, the only thing we can observe is the time it takes to insert objects into a database, and the time it takes to replica a database.

There are three possible tests that can be done:

• Insertion of objects into a database: you can test the time it takes to insert objects into a database. You can specify the size of the objects, the number of objects you want to insert in every transaction, and the frequency of the test. If the database is replicated, you can see the difference in time to update all the images, but this is done in parallel, so the result time doesn’t depend on the number of images, but on the connection speed between the servers.

• Replica of a database: it tests the time it takes to replicate a database into another autonomous partition. It will repeat the test with a determined frequency. The result graphics show the AMS throughput looking also the size of the replicated database.

• Bandwidth: tests the state of the connection between two servers. To start this test, agents must be enabled and the agent server must be running in the remote host. The program will send and agent to the remote server and will send package of a defined size to the agent.

Once you start a test, it stays running until you stop it or you close the federated database or the program. Tests will run in different threads, so too much test running at the same time will give some errors in the measurement.

You can see at any moment the tests that are running in the application, with all the information about them. To visualise the result of a test (even if it is running), click in the ‘Visualise Test’ item in the ‘Statistics’ menu. It will show a graphics bar with the result time of the test.

With it, you can choose the best moment to do some operations with the federated database, when the network is less overloaded, and to see how works a database configuration.

DRO_TOOL includes some new features implemented with the help of agents. You can obtain more information about the host involved in a federated database, like the state of the servers (the AMS server can only be launched and stopped locally), size of the database files and the state of the connection.

If you want more information about agents and how it works, read the agents’ white papers.

To obtain a copy of DRO_TOOL, contact Eva Arderiu or Javier Conde in the IT/ASD group. The application is delivered in a JAR file (Java utility to pack files). This file contains all the classes required by the program, and must not be unpacked.

Once you have the dro_tool.jar file, include it in your CLASSPATH variable. You must specify the path and the name of the file in the CLASSPATH, not just the path.

The program uses the Objectivity/DB for Java bindings so this component must be installed in your system. To check if it is already installed, print the CLASSPATH variable, and the file oojava.jar must be in.

To run the application, call ‘java dro_tool’ from the command line.