Friday, March 16, 2018

Let’s talk about #opsmgr Sizing, baby

The OpsMgr 2012 Sizing Helper is an interactive document designed to assist you with planning & sizing deployments of System Center 2012 Operations Manager. It helps you plan the correct amount of infrastructure needed for a new OpsMgr 2012 deployment, removing the uncertainties in making IT hardware purchases and optimizes cost. A typical recommendation will include minimum hardware specification for each server role, topology diagram and storage requirement.

[source]

IMG_20180316_151010_BokehOne of the issues I have encountered most frequently in the years I have supported customers running System Center Operations Manager is poor performance of their environment because it is under-resourced and over-utilized. And when we have the discussion about resources, I am often told that they used the Sizing Helper tool, so it must be right.

The Sizing Helper really is just that, a helper, a guide. A starting point. It is a guide on the basic hardware to provision for the number of agents you intend on monitoring, without any consideration for additional workloads, number of management packs or environment health. And this is something that is so environment specific, that the calculator will never be able to cater for each scenario.


The Management Pack Lifecycle recommends that you review each management pack you want to import in a test environment before you implement into production exactly for this reason. Your process should look like this:

  1. Download new management pack (or updated version of existing management pack) as well as the management pack documentation.
  2. Read the management pack documentation.
  3. Read it again, this time taking note of the requirements for this management pack, such as run-as account requirements, additional resource requirements, configurations to be made on the target, etc.
  4. Baseline the performance and capacity of your test environment.
  5. Take note of the current workflows on the management servers. You can use Dirk Brinkman’s script to help, if unsure.
  6. Import the management pack into your test environment and configure as per the management pack documentation.
  7. Compare the performance and workflows of the environment before and after import. You will also need to take into consideration how many agents you are monitoring in your test environment vs in production, and identify the load per agent/monitored entity.

I would also recommend you run the management pack in your test environment for at least a two week period, and perform stress testing on the monitored entities. Generate failures, generate event storms, put it through its paces as much as possible to ensure you can gauge, as much as possible, what the impact on your production environment will be.

You will also be able to tune the noise out quite quickly this way, and ensure that, when you import the management pack into production, you are only receiving the alerts you want, you are not over-collecting performance counters you will never use, and you can keep your SCOM environment healthy.

It may sound like a long process, but it will save you a lot of time in the long run.

Friday, February 23, 2018

#OpsMgr 1801 Upgrade issue

caution_warning_sign_sticker-650x800Ran into an issue with the upgrade from OpsMgr 2016 to 1801 with one of my customers recently, and I thought I would share it.

For some reason, they only installed .Net 4 on their management servers, and did not have .Net 3.5 installed.

Turns out the installation requires .Net 3.5 to be installed on the management servers, and, despite the prereq check returning no errors, the installation fails on the Management Server component with the following error reported in the install log:

Error:     :LaunchMSI: MSI C:\System Center Operations Manager\Setup\AMD64\Server\OMServer.msi returned error 1603


The installation then rolls back, which uninstalls all OpsMgr components on the management server.

If you then try to rerun the installation, it will fail on the database selection screen with an error that it cannot connect to the instance/find the database, but when you look in the Install log the error states that the management server already exists in the database and a recovery is not possible at this time.

You will need to recover the database, remove the management server from the management group and reinstall it to recover.

panic-face

So, before you upgrade, make sure you take a full backup of the OperationsManager database, and make sure .Net 3.5 has been installed on all management servers.


ps: I am sorry for shamelessly nicking pictures off the interwebs for this post. If these belong to you, please let me know.

Wednesday, December 20, 2017

#OpsMgr: Extract notification information

One of the areas of OpsMgr that often have the biggest impact on others is the notifications and many customers create a large number of notification subscriptions, which sometimes makes it hard to manage.

I created this PowerShell script to help a customer to extract notification information, covering channels, subscribers and subscriptions. The output of the script is an HTML file saved to the highlighted location


############################################################################
#
#   SCOM-NotificationConfigurationReport-htmlout.ps1
#
#   Script by: Vanessa Bruwer
#
#    This script retrieves all notification configurations and outputs as HTML file.
#
############################################################################

# Variables

$ReportDate = Get-Date -format "yyyy-M-dd"

$rnd = Get-Random -minimum 1 -maximum 1000
$ReportLocation = "C:\Temp"
$ReportName = "$ReportDate-$rnd-SCOM-NotificationConfiguration.html"
$SavetoFile = "$ReportLocation\$ReportName"


# there should be no need to update the script beyond this point

# Delete any previous versions of the report for in case

If (Test-Path $SavetoFile){
     Remove-Item $SavetoFile
}


# Open the content
$body = "<font face=Arial><table border=0><tr><td><b>Notification Configuration</b></td></tr><tr><td><font face=Arial>"


# Connect to OperationsManager

Import-Module OperationsManager

$MGName = (get-scommanagementgroup).Name

# Get Channels

$body += "<table border=1 bordercolor=silver width=100%><tr><td colspan=2><b>Channels</b></td></tr>"
$body += "<tr><td><b>Channel Display Name</b></td><td><b>Type</b></td></tr>"

$Channels = Get-SCOMNotificationChannel

foreach ($Channel in $Channels)
{

$ChannelName = $Channel.DisplayName
$ChannelType = $Channel.ChannelType

#Write-Host $ChannelName, $ChannelType

$body += "<tr><td>$ChannelName</td><td>$ChannelType</td></tr>"

}

$body += "</table>"

$body += "<p></p>"


# Get Subscribers

$body += "<table border=1 bordercolor=silver width=100%><tr><td colspan=2><b>Subscribers</b></td></tr>"
$body += "<tr><td><b>Display Name</b></td><td><b>Address Name</b></td></tr>"

$Subscribers = Get-SCOMNotificationSubscriber

foreach ($subscriber in $Subscribers)
{
$Subscribername = $subscriber.Name

$subscriberaddress = ($subscriber.devices).Name


#Write-Host $Subscribername, $subscriberaddress

$body += "<tr><td>$Subscribername</td><td>$subscriberaddress</td></tr>"

}

$body += "</table>"
$body += "<p></p>"


# Get Subscriptions

$body += "<table border=1 bordercolor=silver width=100%><tr><td colspan=4><b>Subscriptions</b></td></tr>"
$body += "<tr><td><b>Display Name</b></td><td><b>Description</b></td><td><b>Enabled</b></td><td><b>Recipients</b></td></tr>"

$subscriptions = Get-SCOMNotificationSubscription

foreach ($subscription in $subscriptions)
{

$subname = $subscription.DisplayName
$subdescription = $subscription.Description
$subEnabled = $subscription.Enabled
$subrecipients = ($subscription.ToRecipients).Name


#Write-Host $subname, $subdescription, $subEnabled, $subrecipients

$body += "<tr><td>$subname</td><td>$subdescription</td><td>$subEnabled</td><td>$subrecipients</td></tr>"
}

$body += "</table>"

$body += "</td></tr></table></font>"

$body >> $SavetoFile


This is a sample of what the output looks like from my lab environment:

notifications

Monday, December 11, 2017

#OpsMgr: script to assist with moving the OperationsManager and OperationsManagerDW databases

It happens from time to time that we need to move the databases used in our System Center Operations Manager environments from one server to another. While this is a very well documented process for both the OperationsManager database and the Data Warehouse database, it often happens that we miss a step somewhere along the line.

To assist with this process, I have created two scripts, a PowerShell script for updating the registry keys on the management servers and restarting the services, and a SQL script for updating the relevant tables in the database.

I have commented the scripts as much as possible, so it should be fairly easy to figure out what to do. I would recommend running the SQL script first, and then the PowerShell one.

Update the variables (highlighted) before running. Scripts have been tested with OpsMgr 2012 and OpsMgr 2016 in a lab environment. Results may vary in production, and it is highly recommended you test in your lab environment before using in production.

---------------------------------------------------------------------------------------------------------------------

SQL Script:

-- Declare the variable to be used.
DECLARE @OpsDBServer nvarchar(255) = '<server name here>',
         @DWDBServer nvarchar(255) = '<server name here>';


Use OperationsManager

UPDATE MT_Microsoft$SystemCenter$ManagementGroup
SET SQLServerName_43FB076F_7970_4C86_6DCA_8BD541F45E3A = @OpsDBServer


UPDATE MT_Microsoft$SystemCenter$OpsMgrDB$AppMonitoring
SET MainDatabaseServerName_5C00C79B_6B71_6EEE_4ADE_80C11F84527A = @OpsDBServer

UPDATE MT_Microsoft$SystemCenter$DataWarehouse
SET MainDatabaseServerName_2C77AA48_DB0A_5D69_F8FF_20E48F3AED0F  = @DWDBServer

UPDATE MT_Microsoft$SystemCenter$DataWarehouse$AppMonitoring
SET MainDatabaseServerName_5C00C79B_6B71_6EEE_4ADE_80C11F84527A  = @DWDBServer


UPDATE MT_Microsoft$SystemCenter$DataWarehouse$AppMonitoring_Log
SET Post_MainDatabaseServerName_5C00C79B_6B71_6EEE_4ADE_80C11F84527A = @DWDBServer

select SQLServerName_43FB076F_7970_4C86_6DCA_8BD541F45E3A from MT_Microsoft$SystemCenter$ManagementGroup
select MainDatabaseServerName_5C00C79B_6B71_6EEE_4ADE_80C11F84527A from MT_Microsoft$SystemCenter$OpsMgrDB$AppMonitoring
select MainDatabaseServerName_2C77AA48_DB0A_5D69_F8FF_20E48F3AED0F from MT_Microsoft$SystemCenter$DataWarehouse
select MainDatabaseServerName_5C00C79B_6B71_6EEE_4ADE_80C11F84527A from MT_Microsoft$SystemCenter$DataWarehouse$AppMonitoring
select Post_MainDatabaseServerName_5C00C79B_6B71_6EEE_4ADE_80C11F84527A from MT_Microsoft$SystemCenter$DataWarehouse$AppMonitoring_Log


sp_configure ‘show advanced options’,1
reconfigure
sp_configure ‘clr enabled’,1
reconfigure


DECLARE @broker integer

set @broker = (SELECT is_broker_enabled FROM sys.databases WHERE name='OperationsManager')

IF @broker = 0
BEGIN
     ALTER DATABASE OperationsManager SET SINGLE_USER WITH ROLLBACK IMMEDIATE
     ALTER DATABASE OperationsManager SET ENABLE_BROKER
     ALTER DATABASE OperationsManager SET MULTI_USER

    PRINT 'Broker version is incorrect.';
END
ELSE
     PRINT 'Broker version is correct.';

GO


Use OperationsManagerDW

UPDATE MemberDatabase SET ServerName = @DWDBServer

select ServerName from MemberDatabase

GO

---------------------------------------------------------------------------------------------------------------------

PowerShell script:

############################################################################
#
#   SCOM-UpdateDBServerLocations.ps1
#
#
#    This script can be used when moving both the OperationsManager and Data Warehouse
#    databases to a new location.
#    The script will update the registry on all management servers.
#    The script should be run once the databases have been migrated and you are ready
#    to make the final updates.
#    User running this script should have full admin rights on all management servers
#    and update rights on both databases.
#
############################################################################

#Variables

$ManagementServers = "ms1,ms2,ms3,ms4" # add management server names comma separated, this creates an array of the management servers

$stopservices = "Yes" # change to no if services are already stopped


$OpsDBServer = "opsdbservername\instancename,port"

$DWDBServer = "dwservername\instancename,port"


$KeyLocation1 = "HKLM:\SOFTWARE\Microsoft\System Center\2010\Common\Database"
$KeyLocation2 = "HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup"


$ReportDate = Get-Date -format "yyyy-M-dd"

$rnd = Get-Random -minimum 1 -maximum 1000
$ReportLocation = "C:\Temp"
$ReportName = "$ReportDate-$rnd-SCOM-DBMigrationLog.csv"
$SavetoFile = "$ReportLocation\$ReportName"


# there should be no need to update the script beyond this point

# Delete any previous versions of the report for in case

If (Test-Path $SavetoFile){
     Remove-Item $SavetoFile
}

# Open the content
$body = @()

$body += "SCOM DB Migration Log - "+$ReportDate



# Stop services on Management Servers

if ($stopservices -eq "Yes")
{
foreach ($ms in $ManagementServers)
{

    (Get-Service -DisplayName 'System Center Data Access Service' -ComputerName $ms).Stop()
     (Get-Service -DisplayName 'System Center Management Configuration'-ComputerName $ms).Stop()
     (Get-Service -DisplayName 'Microsoft Monitoring Agent' -ComputerName $ms).Stop()

$body += "Services stopped on " + $ms

}
}

# sleep for a few seconds
Start-Sleep -s 20


# update registry key values on MS

$cred = Get-Credential

foreach ($ms in $ManagementServers)
{
$session = New-PSSession -Authentication Kerberos -ComputerName $ms  -Credential $cred -Name RegUpdate

Set-ItemProperty -Path $KeyLocation1 -Name DatabaseServerName  -Value $OpsDBServer
Set-ItemProperty -Path $KeyLocation2 -Name DatabaseServerName  -Value $OpsDBServer
Set-ItemProperty -Path $KeyLocation2 -Name DataWarehouseDBServerName  -Value $DWDBServer

Disconnect-PSSession $session

$body += "Registry updated on " + $ms

}


# Start Services on Managment Services

foreach ($ms in $ManagementServers)
{

    (Get-Service -DisplayName 'System Center Data Access Service' -ComputerName $ms).Start()
     (Get-Service -DisplayName 'System Center Management Configuration'-ComputerName $ms).Start()
     (Get-Service -DisplayName 'Microsoft Monitoring Agent' -ComputerName $ms).Start()


$body += "Services stopped on " + $ms
}


# write-report

$body >> $SavetoFile


---------------------------------------------------------------------------------------------------------------------

As always, copy the scripts into Notepad or similar first and make sure the quotes are not mangled.

Thursday, October 19, 2017

#OpsMgr: Bulk deployment of Linux agents via a script

One of my customers had a requirement to deploy in excess of 2000 Linux agents recently, and was looking for a streamlined way of doing this. I found a couple of scripts online that can help with the deployment of the agent, but none that did quite what we needed to do, so I wrote my own.

Pop the list of agents to deploy into a text file and update the script to point to that file. You can also specify the name of the resource pool that should manage these agents. The script will prompt you for the run-as account username and password when you run it, so this is probably not ready to run unattended.

Script pings agent first to make sure it is reachable, and will then attempt to deploy/configure the agent, the same way the discovery does in the console.

Script results are saved into a CSV file, which is also mailed to you.

Update the bits highlighted in yellow before running. I would also recommend doing to pasting via notepad thing to ensure no mangling of quotation marks, etc happen.


#################################################################
#
#   SCOM-Linux-Agent-Discovery-script.ps1
#
#   Script by: Vanessa Bruwer
#
#
#    This script initiates the discovery for a list of Linux servers from a text file.
#    It is to be used for a set of servers to be managed by the Resource Pool
#    specified in this script.
#
#    The script does some prechecking before the server is added to the discovery.
#
#    It is based on the script provided by Operating Quadrant here:
#    https://operatingquadrant.com/2012/12/06/using-powershell-for-automated-unixlinux-agent-discovery/
#
#################################################################

# Parameters

$UserName = Read-Host -Prompt 'Please enter the Linux username'
$Password = Read-Host -Prompt 'Please enter the Linux password'


# Variables

$ResourcePoolName = "Linux Resource Pool"
$AgentFileLoc = "c:\temp\scripts\linuxagentlist.txt"

$sleeptime = "60"      # sleep time in seconds



$FromEmail = ""
$ToEmail = ""
$MailServer = ""


$ReportDate = Get-Date -format "yyyy-M-dd"

$rnd = Get-Random -minimum 1 -maximum 1000
$ReportLocation = "C:\Temp"
$ReportName = "$ReportDate-$rnd-SCOM-linuxagentdiscovery.csv"
$SavetoFile = "$ReportLocation\$ReportName"



# there should be no need to update the script beyond this point


# Delete any previous versions of the report for in case

If (Test-Path $SavetoFile){
     Remove-Item $SavetoFile
}

# Open the content
$body = @()

$body += "Server Name,Ping Response,DiscoveryResult"


# Read the text file
$Agents = get-content $AgentFileLoc


# Connect to Operations Manager

Import-module OperationsManager

$wsmanuser=$UserName
$wsmanpassword = ConvertTo-SecureString $Password -AsPlainText -Force


$WSCredential = New-Object System.Management.Automation.PSCredential ($wsmanuser, $wsmanpassword)
$Pool = Get-SCOMResourcePool -DisplayName $ResourcePoolName

$SSHCredential = Get-SCXSSHCredential -username $UserName –ElevationType sudo

# Set the counters to Zero
$i = 0
$n = 0

# Loop through the pending agents and ping them
foreach ($Agent in $Agents) {

$n = $n+1

   if (test-Connection -ComputerName $Agent -Count 2 -Quiet )
         {
     $Status = "Responds"

    $DiscResult = Invoke-SCXDiscovery -name $Agent -ResourcePool $Pool -WsManCredential $WSCredential -SshCredential $SSHcredential

    $DiscResult | fl Succeeded, ErrorData
     $DiscResult | Install-scxagent

if ($DiscResult.Succeeded){

$DiscStatus = "Success"
}
elseif ($DiscResult.EndpointAlreadyManaged)
{
$DiscStatus = "Skipped, already managed"
}
elseif ($DiscResult.ErrorData){
$DiscStatus = "Error occurred, please investigate"
}


Start-Sleep -s $sleeptime

         }
          else
          {
       $Status = "Unreachable"
       $i = $i+1

      $DiscStatus = "Skipped"
          }


# Write to the report
$ReportOutput = "$Agent,$Status,$DiscStatus"
$body += $ReportOutput
}


$agentcnt = $n-$i

$body >> $SavetoFile

$mbody = "There are $n Linux agents in the text file, $i not pinging and $agentcnt added to discovery"

Write-Host "There are $n Linux agents in the text file, $i not pinging and $agentcnt added to discovery"

Write-Host “Sending Email”


  #Creating a Mail object
      $msg = new-object Net.Mail.MailMessage
      $att = New-Object Net.Mail.Attachment($SavetoFile)

#Creating SMTP server object
      $smtp = new-object Net.Mail.SmtpClient($mailserver)

#Email structure
  $Subject = "SCOM Linux Agent discovery - $ReportDate"

     $msg.From = $FromEmail
      $msg.ReplyTo = $FromEmail
      $msg.To.Add($ToEmail)
      $msg.subject = $Subject
      $msg.body = $mbody
      $msg.Attachments.Add($att)


   #Sending email
     $smtp.Send($msg)

#write-host "Email sent"


And, as always, test in lab before using in production.

Related Posts with Thumbnails