System Center Operations Manager 2016 High Availability – Configuration

High Availability is an important service for any application and it is highly recommended for a monitoring application. HA solution for a monitoring solution makes sure that the monitoring is always on and the service is available with out interruptions.

From System Center 2012, HA is made easier with the concept of Resource pool, where each member of the pool will synchronize the SQL data and make themselves available during a failure and the same principle applies in System Center 2016 too

Scenarios of HA in System Center Operations Manager

  1. Agent Server fail over to a Management Server from  Resource Pool
  2. Gateway Server Failover to Management Server
  3. Gateway Agent ( domain joined ) Failover
  4. Gateway Agent ( Work-group ) Failover

In order to test this fail-over functionality, I have configured the below servers in my Lab

  • Domain: Kartik.com
  • SCOM Primary Management Server : SCOM2016.kartik.com
  • SCOM Secondary Management Server: SCOM2.kartik.com
  • Gateway Server 1 : Server1.Kartik.com
  • Gateway Server 2 : Node2.kartik.com
  • Domain joined Client Server : Client2.kartik.com
  • Workgroup Computer : Client
  1. Agent Server fail-over to Management Server from a Resource Pool

In this scenario, the agent servers will be reporting to Management Server Resource pool and when one  Management server goes down, the agents reporting to that will fail-over to the other Management Server available in the pool

Test Fail-over

Scenario:

Primary Management Server: SCOM2.kartik.com

Failover Management Server : SCOM2016.kartik.com

Client Server: Client2.kartik.com

1.png

2.png

 

3

Shutdown the Management Server SCOM2.kartik.com to test the agent failover

11.png

SCOM2 showing grey in SCOM console

 

6.png

Event Logs from SCOM2016.kartik.com4

 

Logs from SCOM2016.kartik.com

5.png

Logs from SCOM2016.kartik.com

7.png

Logs from Client2.kartik.com

Here, we see that the server successfully failed over to SCOM2016.kartik.com

9

Client2.kartik.com showing healthy in SCOM console

10

 

2. Gateway Server Fail-over

Gateway Server: Server1.kartik.com

Primary Management Server: SCOM2.kartik.com

Failover Management Server: SCOM2016.kartik.com

 

13

  • Powershell Commands to configure Gateway Server failover

 

$primaryMS = Get-SCOMManagementServer –Name “SCOM2.kartik.com”

$failoverMS = Get-SCOMManagementServer –Name “SCOM2016.kartik.com”

$gatewayMS = Get-SCOMGatewayManagementServer –Name “Server1.kartik.com”

Set-SCOMParentManagementServer –Gateway $gatewayMS –PrimaryServer $primaryMS

Set-SCOMParentManagementServer –Gateway $gatewayMS –FailoverServer $failoverMS

14.png

Powershell Commands to verify Gateway Server Fail-over 

$GWs = Get-SCOMManagementServer | where {$_.IsGateway -eq $true}

$GWs | sort | foreach {

       Write-Host “”;

       “Gateway MS    :: ” + $_.Name;

       “–Primary MS  :: ” + ($_.GetPrimaryManagementServer()).ComputerName;

       $failoverServers = $_.getFailoverManagementServers();

       foreach ($managementServer in $failoverServers) {

              “–Failover MS :: ” + ($managementServer.ComputerName);

       }

}

Write-Host “”;

15.png

Verify Gateway Server Fail-Over

Shutdown the primary management Server SCOM2.kartik.com

Logs from SCOM2016.kartik.com

16.png

Event generated in SCOM console for SCOM2.kartik.com

17

Logs from Server1.kartik.com saying that it is successfully failed over to SCOM2016.kartik.com

18

Server1.kartik.com showing healthy in SCOM console

 

19

3. Gateway Agent ( domain-joined ) failover

Client: Client2.kartik.com

Primary Gateway Management Server: Server1.kartik.com

Failover Gateway Management Server: Node2.kartik.com 20

Client2.kartik.com reporting to Gateway Server1.kartik.com

21

 

Powershell commands to configure Gateway Agent failover

$primaryMS = Get-SCOMManagementServer | where {$_.Name –eq ‘server1.kartik.com’} 
$failoverMS = Get-SCOMManagementServer | where {$_.Name –eq ‘Node2.kartik.com’} 
$agent = Get-SCOMAgent | where {$_.PrimaryManagementServerName -eq ‘Server1.kartik.com’} 
Set-SCOMParentManagementServer -Agent: $agent -PrimaryServer: $primaryMS 
Set-SCOMParentManagementServer -Agent: $agent -FailoverServer: $failoverMS

22.png

Powershell commands to verify Gateway Agent failover

 

$Agents = Get-SCOMAgent | where {$_.PrimaryManagementServerName -eq ‘Server1.Kartik.COM’} 
$Agents | sort | foreach { 
Write-Host “”; 
“Agent :: ” + $_.Name; 
“–Primary MS :: ” + ($_.GetPrimaryManagementServer()).ComputerName; 
$failoverServers = $_.getFailoverManagementServers(); 
foreach ($managementServer in $failoverServers) { 
“–Failover MS :: ” + ($managementServer.ComputerName); 


Write-Host “”;

23.png

Shutdown Server1.kartik.com

Event generated in SCOM console for Server1.kartik.com

24.png

Event Log from Management Server SCOM2016.kartik.com

25.png

Client2.kartik.com successfully failed over to other gateway server Node2.kartik.com

Event log generated in Client2.kartik.com

26.png

Client2.kartik.com showing healthy in scom console

28.png

4. Gateway Agent ( workgroup ) failover

Workgroup computer: Client.kartik.com

Primary Gateway Management Server: Server1.kartik.com

Failover Gateway Management Server: Node2.kartik.com

27.PNG

Note: For the workgroup computer to failover , the certificate used for client authentication should be imported into personal store of failover Gateway Management Server too

Workgroup client reporting to the gateway Server1.kartik.com

31.png

Certificates imported in personal store of both the Gateway Servers Server1.kartik.com and Node2.kartik.com

29.png

Powershell commands to verify Gateway Agent failover

30.png

Shutdown Server1.kartik.com

32

Event logs generated from Management Server SCOM2016.kartik.com

33

Event Log generated in workgroup computer for successful failover

34

 

 

 

 

Advertisements

Powershell Script to clear cache on SCOM Agents

 

$path = “C:\GreyAgents.txt”

$srvlist = Get-Content “$path”

$serviceName = “HealthService”

Foreach ($srv in $srvlist)
{
Write-host “Greyagents” : “$srv”

Invoke-Command -ComputerName $srv -Scriptblock{ Stop-Service -ServiceName ‘HealthService’}

Invoke-Command -ComputerName $srv -Scriptblock{ Remove-item -path “C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State” -Recurse}

Start-sleep -Seconds 10

Invoke-Command -ComputerName $srv -Scriptblock{ Start-Service -ServiceName ‘HealthService’}

Write-host “Cleared Cache Successfully”

}

 

 

Powershell Script to schedule Maintenance Mode in SCOM

 

$path = “C:\SCOMMaintenanceMode.txt”
$domain = “kartik.com”

 

$MyFile = Get-content “$path”
$MyFile
foreach($srv in $MyFile)
{
Write-host “ServerName : $srv”

$startTime = [DateTime]::Now
$endTime = $startTime.AddMinutes(20)

$srv += “.$domain”

$Class = get-SCOMclass | where-object {$_.Name -eq “Microsoft.Windows.Computer”};
$Instance = Get-SCOMClassInstance -Class $Class | Where-Object {$_.Displayname -eq “$srv”};
Start-SCOMMaintenanceMode -Instance $Instance -Reason “PlannedOther” -EndTime $endTime -Comment “Scheduled SCOM Maintenance Window”

}

 

Powershell Script to recycle HealthService on all GreyAgents in SCOM

$path = “C:\GreyAgents.txt”

$srvlist = Get-Content “$path”

$serviceName = “HealthService”

Foreach ($srv in $srvlist)
{
Write-host “Greyagents” : “$srv”

 

Invoke-Command -ComputerName $srv -Scriptblock{ Stop-Service -ServiceName ‘HealthService’}

 

Start-sleep -Seconds 10

 

Invoke-Command -ComputerName $srv -Scriptblock{ Start-Service -ServiceName ‘HealthService’}

Write-host “Health Service ReStarted Successfully”

}

 

List out Grey agents in SCOM with Powershell

# Create a file for output

$file=”C:\Greyagents.txt”

$startdate =Get-date

$runtime =”$(Get-date -format “M/dd/yyyy H:MM”)”

$CurrentDate = $CurrentDate.ToString(‘MM-dd-yyyy_hh-mm-Ss’)

#get the SystemCenter Agent Class

$agent = Get-SCOMClass | where-object{$_.name -eq “microsoft.systemcenter.agent”}

#Get the grey agents

$objects = Get-SCOMMonitoringObject -class:$agent | where {$_.IsAvailable –eq $false}

forEach($object in $objects)
{

# display list of grey agents in PS window

write-host “Greyagents:$object”

#if you want output to Notepad, execute this

$object.displayname+”,”+$Object.HealthState| Out-file $file -append

# if you want output to csv, execute this

$object|Select Displayname,Healthstate | Export-Csv -Path “C:\Greyagents\Greyagents_$currentdate.csv”

}

 

 

 

 

 

 

Useful Excel Tips

How to compare two columns in Excel

 

  1. Select the specific column
  2. Navigate to Home- Conditional Formatting
  3. Select New Rule
  4. Select the option ” Use a formula which cells to format
  5. Add the formula =countif($B:$B, $A1)
  6. Navigate to Format Cells and select Fill
  7. Select any colour to differentiate the results
  8. Now the matching values will have the selected colour

Useful Powershell commands for System Center Operations Manager

  1. Export Management Packs :

To Export all the available Management Packs

Export-SCOMManagementPack -Path “C:\MPArchive”

To Export the List to CSV

Get-SCOMManagementPack | Export-CSV C:\MP.csv

To Export a Specific Management Pack

Get-SCOMManagementPack -Name *ManagementPack Name* | Export-SCOMManagementPack -Path “C:\MPArchive”

To Export Monitors of a Specific Management Pack 

Get-SCOMManagementPack -Name *ManagementPack Name* | Get-SCOMMonitor | Export-csv spmonitor.csv

To Export Rules of a Specific Management Pack 

Get-SCOMManagementPack -Name *ManagementPack Name* | Get-SCOMRule | Export-csv sprule.csv

 

To get disabled discoveries in a Management Pack

Get-SCOMManagementPack -Name *sharepoint* | where-object {$_.sealed -eq $false} | export-csv disableddiscoveries.csv

 

To Get the critical errors in SCOM for a particular period

Get-SCOMAlert | Where-Object{$_.Timeraised -gt “5/13/2017”} | where-Object{$_.Severity -eq “error”} | measure | export-csv Warnings.csv

 

To get the list of all rules/Monitors from all Management Packs

Get-SCOMMonitor | select DisplayName, ManagementPackName, Enabled, ManagementGroup | Export-csv ConsolidatedMonitors.csv

To get the ManagementServer Name to which the Agent/Gateway Server reports to

Get-SCOMGatewayManagementServer |Where {$_.DisplayName -eq “GatewayServerFQDN”} |Get-SCOMParentManagementServer

Get-SCOMAgent |Where{$_.DisplayName -eq “AgentFQDN”} |Get-SCOMParentManagementServer