Using Wild card in File Watcher [ctmfw ]

Most of us know that ctmfw utility is used for File Watcher (create/delete). Normally most of us (all of us.. may be ???? I've used many times before knowing the exact usage ;-D ) will use the file name in quotes (" "), which is actually wrong. The correct way of using this is , placing the quotes only with the file name not with entire path.

Invalid: "test/mahi/touchfile.txt"

Valid: test/mahi/"touchfile.txt"

This will be very helpful while using wildcard character (*)

Eg: test/mahi/"touchfile*.txt"

This will give a flexibility to watch for any file like touchfile1.txt, touchfile2.txt, touchfile-random.txt... etc

Read more »

Control-M Road Map [ History of Control-M ]

Control-M was a batch-scheduling tool originally developed for scheduling jobs on mainframe computer systems, by an Israel-based company New Dimension Software (the company was originally called 4th Dimension Software and later renamed to New Dimension Software). In the early 90's, Control-M also became available on the distributed environment (for example, Windows and Unix). Users can use Control-M either in a distributed/mainframe only or mixed environment. There are a number of other mainframe-focused products within the same product line such as Control-D, Control-M/Tape, Control-O, Control-V, and Control-M/Links. BMC software acquired New Dimension Software in 1999 and continued the product line. In the recent years, BMC software made Control-M to become part of its BSM strategy by integrating Control-M with BMC Atrium Configuration Management Database (CMDB). Today Control-M is one of the most widely used enterprise batch scheduling software on both mainframe and distributed environments, with customers around the world from every major industry.

The following are the milestones in the Control-M history:

• 1985: Innovated the integrated approach for batch processing automation (mainframe)
• 1993: Provided single focal point of control of the heterogeneous batch environment – Enterprise Control Station (later on, the name changed to Control-M/Enterprise Manager)
• 1993: Release of control modules to extend Control-M's scheduling capability into external applications
• 1996: Enabled event-based scheduling
• 2000: Release of new control modules - integration with ERP applications
• 2004: Added the support for SOA approach – Business Process Integration
• 2004: Link batch scheduling with business SLAs – Batch Impact Manager
• 2006: Release of Agentless technology - part of Control-M 6.3.01
• 2007: Integration with BMC Atrium CMDB – BMC Batch Discovery
• 2008: Workload Lifecycle Management - within Control-M 6.4.01
• 2010: Workload automation – Control-M 7
• 2012: The Power of Simple – Control-M 8

Reference: BMC Control-M 7: A Journey from Traditional Batch Scheduling to Workload Automation

Read more »

May be Conditions (Manual Conditions)

Sometimes we will get different scenarios in Scheduling control-M jobs, one of that is like below.

Lets say there are JOB-A,JOB-B,JOB-C. JOB-C, is dependent on other 2 jobs.

• If JOB-A & JOB-C are DAILY Jobs
• JOB-B is Mon, Wed, Fri job 

If you try to implement the above change the first thing comes to your mind is using DUMMY job (in case if you are good in Scheduling though).

if it is a single scenario in your entire batch jobs then we can use Dummy jobs, lets say you have 100's of similar scenarios. then it would be difficult to have 100's of dummy jobs, it will make your scheduling solution complex, so it is better to use May Be Conditions or Manual Conditions.

Steps to Implement this:

1. Out Conditions of JOB-A and JOB-B should be  #-JOB-A-OK and   #-JOB-B-OK
2. In  Conditions of JOB-C is  #-JOB-A-OK and   #-JOB-B-OK
3. Create 1 job with TASK TYPE as Command and use "ctmldnrs -CALCCOND -ADDMODE NO" as a commad
4. Create 1 job with TASK TYPE as Command and use ctmldnrs -LOAD "#-*" as a commad

The job in Step3 will create a default output file in <controlm_user_dir/ctmldnrs.dat>

The job in Step4 will read the conditions from a default file <controlm_user_dir/ctmldnrs.dat> and load it on Conditions table. This means the jobs which are not in schedule for that day and if there are any jobs waiting for their conditions (#- Conditions) will be created in Conditions table.

Note:

• Step3 job must run immediately after New Day Process

• Step4 job must run after Step3 job

I hope it is clear. Please drop your comments in case if you have any queries.

Read more »

BMC Batch Impact Manager

Useful Video to know about BMC Batch Impact Manager

Read more »

Recovering from primary CONTROL-M/Server failure (Server failover)

These procedures assume that a failover server has already been installed and initialized on a secondary computer. Two procedures are used during failover situations:

1. Verify that:
• Both primary and secondary CONTROL-M/Server installations and CONTROL-M/Server Configuration Agent are not running.
• Both the primary and mirror database servers are running.
2. Log on to the secondary computer as the CONTROL-M/Server owner.
3. Specify the ctm_menu command to open the CONTROL-M Main Menu.
4. Type 4 to select Database Mirroring and press <Enter>.
5. From the Database Mirroring menu, type 7 to select Start Failover and press <Enter>.

WARNING


Do not start the secondary CONTROL-M/Server while the primary CONTROL-M/Server is running, or before the Start Failover procedure finishes executing. If the secondary CONTROL-M/Server is started too early, the following message may be displayed: 

** Unable to start CONTROL-M due to path specified for PROCLIB directory $PROCLIB. This path is invalid **

6. The following message is displayed:

This operation disables CONTROL-M database mirroring, and starts FailOver CONTROL-M !!!

Confirm that CONTROL-M in the primary data center is not running.Continue [y/n] :

7. Type y and press <Enter> to continue with Failover.

The following message is displayed:
CONTROL-M/Server database mirroring 

Mirroring of the CONTROL-M/Server database allows rapid recovery of CONTROL-M/Server functionality in the event of a failure of the CONTROL-M/Server database. You can implement mirroring during or after CONTROL-M/Server installation.

To implement CONTROL-M/Server database mirroring

1. Review the database requirements for mirroring as described in the CONTROL-M/Server for UNIX and Microsoft Windows Installation Guide. 


2. Verify that CONTROL-M/Server, CONTROL-M/Server Configuration Agent, and CONTROL-M/Server utilities are not running. Ensure that the SQL database is running.


3. Choose Database Mirroring => Initialize Mirroring from the CONTROL-M Main Menu.


4. At the conclusion of the procedure, restart CONTROL-M/Server and the CONTROL-M/Server Configuration Agent

Read more »

Control-M/EM GUI Authorizations

To perform any action in Control-M we need Authorization/Permission. This will be setup using Authorizations window in Control-M/EM GUI. Using this we can create, modify or Delete user-id's and their privileges. This is the window Schedulers or Admin will use to create a User-Id or Reset Password or modifying privileges.. etc.

To access the Authorizations window Follow the below steps:

1. Log on to CONTROL-M/EM GUI or CONTROL-M/Desktop.
2.  Choose Tools => Authorizations. The Users tab is displayed.

The Authorization window has separate panels for users and groups. The Users panel enables you to add, modify, copy, or delete a user whose authorizations can be configured. The Groups panel enables you to add, modify, copy, or delete a group whose authorizations apply to members of the group.

Actions that are permitted or denied for each access level are described below. 

1. None - Denies access to the component.  
2. Browse - Can view and refresh the component.  
3. Update - Permits adding and modifying CONTROL-M information.  
4. Full - Permits adding, modifying, and deleting CONTROL-M information.  
5. Default - [Valid for user authorizations. Not valid for group authorizations.] Inherits the permissions from the group to which the user belongs. Use of this is, We can create a Group with list of Privileges

The panel will look like below.

Authorizations window – Users tab fields

• Filter: Select this check box to enable the User Name filter.
• User Name text box: Specify a user name or pattern (for example, adm*). Click Apply to apply the filter. Click Clear to erase the contents of the text box.
• Name: CONTROL-M/EM logon name of the user to whom the authorization refers. Names listed satisfy the requirements of the filter, if any.
• Full Name: Full name of the user.
• Description: Description of the user.

Authorizations window – Groups tab fields

• Filter: Select this check box to enable the Group Name filter.
• Group Name text box: Specify a group name or pattern (for example, adm*). Click Apply to apply the filter. Click Clear to erase the contents of the text box.
• Name Group name: Groups listed satisfy the filter requirements, if any.
• Description: Description of the group.

User Properties Window:

Authorizations window – User Properties fields

• General: Provides logon data about users (or lists members of a group).
• Member Of: [User Authorizations window] Matches users with groups.
• Jobs: Jobs a user (or member) can access and actions he can perform.
• Privileges: 
• Whether the CONTROL-M Configuration Manager can be accessed and used, Alerts and Archived Viewpoints can be viewed, and the user (or member) can use the Command Line Interface (cli).
• Actions that can be performed on Collections, Hierarchies, Filters, ViewPoints, and Authorization screens.
• Scheduling Tables: Scheduling tables a user (or member) can access in CONTROL-M/Desktop, authorization levels for the tables, and whether jobs or tables can be ordered or forced.
• Prerequisite Conditions: Prerequisite conditions that a user (or member) can access.
• Control Resources: Control resources that a user (or member) can access.
• Quantitative Resources: Quantitative resources that a user (or member) can access.
• Global Conditions: Global conditions that a user (or member) can access.
• Calendars: Calendars that a user (or member) can access.
• Owners: Jobs in scheduling tables that a user can access.

Read more »

Recovering from Database Failure

If the primary database fails, two options exist for restoring CONTROL-M/Server operation:

1) Operate CONTROL-M/Server using the mirror database instead of the primary database. The primary CONTROL-M/Server database can be rebuilt at a later stage.

This method resumes CONTROL-M/Server operation in the minimum time possible. However, mirroring cannot be reactivated until after the primary database is repaired.

2) Restore the CONTROL-M primary database from the mirror database and start CONTROL-M/Server again.

This method allows the CONTROL-M/Server database to continue to operate with mirroring. However, you will not be able to start CONTROL-M/Server until you have completed restoring the CONTROL-M/Server database - and this can result in significant downtime.

Read more »

Failover Server

A Secondary installation of the CONTROL-M/Server resides on a separate computer. Typically the database of the secondary CONTROL-M/Server, created as part of the CONTROL-M/Server installation, acts as a mirror database.

The secondary CONTROL-M/Server, including its database, acts as a backup to the primary CONTROL-M/Server. This secondary CONTROL-M/Server computer is sometimes called the Mirror Environment.

The secondary CONTROL-M/Server duplicates all of the primary CONTROL-M/Server’s links to CONTROL-M/Agents, and CONTROL-M/Server Configuration Agent. The secondary CONTROL-M/Server is defined in each agent’s list of authorized server hosts.

During normal operation, the secondary database is constantly updated to remain identical with the primary database. The secondary CONTROL-M/Server remains inactive.

If the primary CONTROL-M/Server fails due to a hardware/software problem on the primary computer, the administrator activates failover on the secondary CONTROL-M/Server. The secondary CONTROL-M/Server performs all normal CONTROL-M/Server functions until the primary CONTROL-M/Server is repaired.

Jobs submitted to agent computers before failover continue executing. The secondary CONTROL-M/Server polls the agent computers to determine the status of jobs listed in the Active Jobs file.

When the primary CONTROL-M/Server is stable, normal operation is restored. The failover server is shut down, the primary database is restored, and the primary CONTROL-M/Server is restarted. The secondary server returns to passive readiness.

Implementing a Failover Server

A failover server can be implemented any time after CONTROL-M/Server has been installed.

The secondary CONTROL-M/Server installation should be built on a separate computer and database server, independent of the computers that host the primary CONTROL-M/Server and its database.

Several administrative functions are used to initialize and use the failover server, and restore the primary computer. These functions are accessed from the Database Mirroring menu of the CONTROL-M/Server Main Menu.

Preparing a Failover server

Verify the following conditions for the secondary installation:

The Agent-to-Server Port Number and Server-to-Agent Port Number communication parameters must be the same on both CONTROL-M/Server computers. These ports must also be dedicated to the use of CONTROL-M/Server. You should also ensure that both computers are using the same date.

1. The failover server can be configured to use a dedicated or a non-dedicated SQL database server. However, it should not use the same instance of the database server used by the primary environment.


2. The Database (Data Portion) Size parameter should be assigned the same value as the current size of the primary database.


3. All agent computers that are defined on the primary CONTROL-M/Server should also be defined on the failover server. These definitions must be identical. If you installed CONTROL-M/Agent on the primary CONTROL-M/Server computer, it should also be installed on the failover server. The parameters should be the same on both CONTROL-M/Server installations.

The host name of the failover server must be added to the list of authorized hosts of each agent.

Read more »

Mirror Database

In addition to the primary database installed with CONTROL-M/Server, a secondary database is built on another Database Server computer. All updates to the primary CONTROL-M/Server database are written to the secondary mirror database.

If updates to the mirror database fails, the mirror database is considered damaged and writing to it is disabled. Normal CONTROL-M/Server operation is not affected. Mirroring can be re-initialized after you repair the communications link.

If the CONTROL-M/Server primary database server fails, or if the primary database loses integrity, CONTROL-M/Server operation can be restored using either of the following manual interventions:

You can direct CONTROL-M/Server to use the mirror database instead of the primary database. The primary CONTROL-M/Server database can be rebuilt when time allows.

This method ensures that CONTROL-M/Server functionality is resumed as soon as possible. However, mirroring is not resumed until after you repair the failed primary database and restore it from the mirror database.

You can repair and rebuild the CONTROL-M/Server primary database and restore it from the mirror database.

This method allows mirroring of the CONTROL-M/Server database to continue without interruption. However, depending on database size and the cause of the failure of the database, it may take some time to rebuild the CONTROL-M/Server database before CONTROL-M/Server can be restarted.

Implementing a Mirror Database

Database mirroring can be initialized any time after CONTROL-M/Server has been installed.

The mirror database should be built on a separate database server, independent of the database server that hosts the primary CONTROL-M/Server database.

Several administrative functions are used to initialize, enable, and restore CONTROL-M/Server database mirroring. These functions are accessed from the Database Mirroring menu of the CONTROL-M/Server Main Menu.

Read more »

Failover Planning

As CONTROL-M is integrated in the production environment of the data center, it becomes increasingly important to ensure that interruptions of CONTROL-M functionality are as short as possible.

A properly designed and executed failover plan ensures that CONTROL-M functions are resumed as soon as possible if a recovery is necessary.

Failover planning for CONTROL-M should provide for the following contingencies:

1. Loss of integrity in the CONTROL-M/Server primary database due to a failure of the SQL server that maintains the database


2. Failure of the computer on which CONTROL-M/Server runs (server computer) Ideally, your failover plan should include a complete backup for both the server computer and the CONTROL-M/Server database.

You can implement either one of the following levels of failover capability:

Mirror Database: a database on an independent database server mirrors the CONTROL-M/Server database. If this level of failover is implemented, only CONTROL-M/Server database is backed-up and will be available in case of loss. There is no failover support for any loss of information affecting the CONTROL-M/Server installation. Readmore...

Failover Server: a secondary CONTROL-M/Server installation on a different computer. The CONTROL-M/Server database of the secondary installation is considered the mirror database. If this level of failover is implemented, failover support is provided for loss of information affecting both the CONTROL-M/Server installation and the CONTROL-M/Server database. Readmore...

Read more »