KB: Understanding Operations Manager Maintenance Mode

What is Maintenance Mode?

Maintenance mode is a feature in Operations Manager to suspend monitoring of an object during regular software or hardware maintenance activities like Software Update installations, Hardware replacements, etc.

What happens when an object is put into Maintenance Mode

When an object is placed into Maintenance Mode, all workflows targeted against that object are suspended during that specific interval. This prevents Alerts and Notifications from being generated by the workflows targeted against the object in Maintenance Mode and helps reduce alerting noise during a planned maintenance activity on the server. In Maintenance Mode, alerts, notifications, rules, monitors, automatic responses, state changes and new alerts are suppressed at the agent.

The following event is logged on the Operations Manager event log during starting and ending of Maintenance Mode for a monitored object:

=============================================

Log Name: Operations Manager
Source: HealthService
Date:
vent ID: 1216
Task Category: Health Service
Level: Information
Keywords: Classic
User: N/A
Computer: Server.Domain.COM
Description:
Resuming monitoring for instance "Database" with id:"{GUID}" as the instance maintenance mode is OFF. Management group "MG".

=============================================


IMPORTANT NOTE:
It is not recommend that you put management servers into Maintenance Mode. Configuration distribution, the heartbeat feature, and other features for the system might become unreliable. If you place a management server into Maintenance Mode, alerts, notifications, rules, monitors, automatic responses, state changes, and new alerts generated on the management server will be suppressed. The health service on the management server continues to run. Alerts, notifications, rules, monitors, automatic responses, state changes, and new alerts from other agent-managed computers will be processed and displayed as appropriate.

How to put an object into Maintenance Mode

In Operations Manager, Monitoring is based on specific classes. A specific monitored object on an agent can be put into Maintenance Mode instead of putting the entire Computer into Maintenance Mode.

In the example below, the SQL Database Engine running on Server A will be updated to the current version. This software update maintenance is expected to take 30 minutes to complete. During this time, the default instance of the SQL Database engine running on this server will not be available.

In this case, we can put the SQL Database engine and its entire contained object into Maintenance Mode instead of putting the entire server into Maintenance Mode. This helps in monitoring the Operating System and other applications running on the server while SQL maintenance is performed.

The default instance of SQL running on Server A can be put into Maintenance mode as follows (Note that this article assumes that we have a working Operations Manager management group with the SQL management pack already installed and working):

1. Open the Operations Manager console. Under the Monitoring Pane, expand the folder named “Microsoft SQL Server”, then expand “Server Roles” and select the “Database Engines” State view.

2. This will list all monitored instances of SQL Database engine in this management group.

3. Select and right-click the default instance (MSSQLSERVER) whose Path value is Server A. In the Displayed menu, select ”Maintenance Mode”.

Choosing “Selected objects only” Puts only the Database engine into Maintenance Mode.

Choosing “Selected Objects and all their contained objects” puts the Database Engine and all the databases hosted by this instance of SQL database engine into Maintenance Mode.

An easier way to find the contained objects is to open the Diagram view for the specific object and drill down in the view.

You can select the Time interval for the Maintenance Mode using on the options under the “Duration” as required.

How to Schedule Maintenance Mode

Operations Manager does not allow scheduling Maintenance Mode for monitored objects from the console. However, Maintenance Mode can be scheduled for a future time with the help of the Maintenance Mode schedule tool that is part of the Operations Manager R2 Admin resource kit. Please see the link below for more information:

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=26139

NOTE This tool does not work with System Center 2012 Operations Manager

The blog below from the Operations Manager community also has a tool for scheduling Maintenance Mode for monitored objects:

http://www.scom2k7.com/scom-remote-maintenance-mode-scheduler-2-0-r2-scripts/

Additionally, you can use Orchestrator to put objects into Maintenance Mode on a schedule, and there are also third party tools/scripts available.

How to automate starting and Ending Maintenance Mode using SCOM SDK

The link below provides information on how to automate starting and ending Maintenance Mode of specific objects using Operations Manager 2007 R2 SDK:

http://msdn.microsoft.com/en-us/library/bb437532.aspx

How to view the objects that are in MM

To view all objects of a specific class that are in Maintenance Mode using the Operations console, follow the steps below.

1. Open Operations console. Under the Monitoring pane create a state view named “Maintenance Mode” under the specific unsealed Management Pack folder or under the “Monitoring” folder at the root of the view tree. Creating the View under the “Monitoring” folder at the root of the view tree saves the view in the Default Management Pack.

2.Under the Name option, Type in “Maintenance Mode”

Change the Value under “Show data related to” from Entity to the Class of objects that you want to view.

Select “is in Maintenance Mode” check box under the “Select conditions”

Click Ok to create the view.

This view will display all objects of that specific class that are in Maintenance Mode currently.

Maintenance Mode Cmdlets

The following command lets are available for Managing Maintenance mode using the Operations Manager command shell in Operations Manager 2007 R2:

New-MaintenanceWindow – This cmdlet puts the Specified monitoring object into Maintenance Mode. More information about this cmdlet is here:

http://technet.microsoft.com/en-us/library/gg132214.aspx

Set-MaintenanceWindow – This cmdlet sets properties of a maintenance windows. It can be used to update end time, reason and comments on the current Maintenance Mode for the monitoring object specified by the Monitoringobject parameter.

More information about this cmdlet is here :

http://technet.microsoft.com/en-us/library/gg132242.aspx

Get-MaintenanceWindow – This cmdlet gets maintenance window information for a monitoring object that is currently in Maintenance Mode. The cmdlet can retrieve maintenance windows for top-level items, items at the current location or for specified monitoring objects.

More information about this cmdlet is here :

http://technet.microsoft.com/en-us/library/gg132195.aspx

How maintenance mode helps reporting

Starting and ending the Maintenance Mode for a specific object for a planned maintenance helps the Operations Manager administrator report accurately on the planned and unplanned downtime of an application or a service. This helps report the availability of a specific service or an application or a server accurately during a specific period of time.

Maintenance Mode Known issues:

1. A computer agent unexpectedly generates heartbeat alerts after you put it into Maintenance Mode in System Center Operations Manager 2007

This issue has been explained in the below KB article:

http://support.microsoft.com/kb/942866

2. Configuration Manager and Operations Manager agent Maintenance Mode

System Center Configuration Manager 2007 supports Maintenance Mode for Operations Manager with its Software Distribution and Software Updates features. By placing the Agent into Maintenance Mode, alerts can be suppressed on computers running the Configuration Manager client.

This option works well if running MOM 2005 but does not operate as you may think with Operations Manager 2007 and later. When you set this option and Configuration Manager attempts ‘Maintenance Mode’ on an Operations Manager agent, the result is that the health service is paused. While the health service is paused you may get heartbeat failure errors, and when the health service is resumed all queued up actions will process at that time resulting in potential alerts. For this reason and others, we do not recommend using the Configuration Manager option for Operations Manager agents.

3. Alerts are generated by an agent while in Maintenance Mode

This issue could be caused due to the following reason:

- The object that was put into Maintenance Mode is not generating the actual alert. To verify this, right-click “Active Alerts” view on the Operations Manager console and select “Personalize view” from the menu. Select the check box next to “Class” under the “Columns to display” option.

You will see a new column “Class” under the Active Alerts view. Make sure the instance of this class for that agent is in Maintenance Mode.

- The alert was generated before monitoring was suspended on the agent. This can be verified by checking if the Event ID 1215 confirming the monitoring of the specific instance is suspended.