First published on TECHNET on Jun 07, 2012
One of the major pieces of functionality in Orchestrator and in many of the Integration Packs the ability to monitor something (a database, a server) and start a runbook when a certain condition occurs. The monitor activities are used extensively by customers in a wide array of scenarios. But is it the best way to accomplish the scenario?
First of all, let’s take a look at how runbooks are handled by Runbook Servers. Each running runbook consumes a “slot” in the available runbook throttling limit that’s set for each Runbook Server. By default, this limit is set to 50, but in reality you might set that much higher depending on the types of load your runbooks place on the system, the performance characteristics of the server, and so on (for more info on changing the limit, see TechNet:
How to Configure Runbook Throttling
). Regardless of what the limit is or whether you will run enough runbooks concurrently to reach it, there is a limit. For every runbook that is running - every monitor runbook that is active, every runbook and all of the child runbooks that they might call – there is a PolicyModule.exe instance and that is consuming one of those slots in the throttling limit.
In addition to the above, realize that you may have multiple monitor runbooks running against the same data source. For example, you may be monitoring Operations Manager for certain conditions like new alerts of a certain type or changes in status of a monitor for a set of computers. It’s highly likely that you will have multiple monitors, each polling the Operations Manager server every 5 seconds, putting additional load on the Operations Manager server because it has to do queries of its data every 5 seconds for 5 or 10 (or maybe more) different monitors. Now repeat this scenario across your other data sources, like Service Manager, Configuration Manager, Virtual Machine Manager, and so on. You begin to see how this might have a performance impact not only on Orchestrator, but on the other systems as well. I hope you also see the answer to the question in the title
The better solution, when you have the ability to implement it in the data sources outside of Orchestrator, is triggering. By that I mean actively invoking a runbook to start when the outside system, such as Operations Manager, sees a specific condition occur. For example, Operations Manager is highly tuned for performance in gathering monitoring data and triggering actions based on that data. It is a much more optimal solution to add an additional action to its list of tasks to perform on given conditions. In the case of a monitor, when Operations Manager is monitoring a system and detects a failure condition, instead of just issuing an alert, it could also trigger a runbook to start and even pass it parameters. Using a simple tool like the
command-line Job Runner utility
, you can perform a remediation task that is an Orchestrator runbook. Not only are these more flexible than locked Management Packs, they have a lot more capabilities in crossing multiple IT silos and connecting to many more systems outside the reach of Operations Manager. You can also integrate the runbook to coordinate the actions with Service Manager or any other application for which there is an Integration Pack.
What about those systems where you don’t have the flexibility to add actions to invoke a runbook? Well, for those situations, using monitor runbooks is still a viable option, but depending on the number of individual monitors you expect to have, you might want to try aggregation of monitors into a sort of “publisher-subscriber” model. This kind of model combines the queries needed by multiple runbooks into a larger, aggregated query, so that there is one (or at least fewer) individual query that is polling the data source. Then, based on the data received, you set conditional links that then trigger the other runbooks needed to perform the required actions. The resulting runbook might look something like this:
You would just set conditions on the link based on the differences in the data that determined what runbook to invoke, such as the computer name or monitoring class, or state. Using this methodology aggregates the polling being performed on the remote data source and not only improves performance on the Orchestrator server and frees up slots in the throttling limit, it improves performance on the remote server because it doesn’t have as many individual monitors polling it constantly.
A better publisher-subscriber model would allow you to simply have a single monitor in a runbook and a database table behind it that stores the subscribers and the data conditions they need to activate, rather than having to add lots of Invoke Runbook activities and keep those up to date. But, that’s a custom activity for another time…