How to set failed workflow instances as ignored from PS

December 6, 2024 Posted by Alexander Axberg

$scsmMgmtServer = "scsmMGMTServerHere"
$emg = New-Object Microsoft.EnterpriseManagement.EnterpriseManagementGroup $scsmMgmtServer

#MONSTER WORKFLOW LIST
$wfSubs = $emg.Subscription.GetSubscriptionsByCriteria("Name LIKE '%'")

foreach ($wfSub in $wfSubs)
{
    $wfSubFailedInstances = $emg.Subscription.GetFailedSubscriptionStatusById($wfSub.id) | ?{$_.status -eq "Failed"}
    foreach ($subFailedInstance in $wfSubFailedInstances)
    {
        #Ignore
        $emg.Subscription.IgnoreFailedSubscription($subFailedInstance)
    }
}

Source: https://community.cireson.com/discussion/2581/workflows-arent-getting-created-run-and-scsm-console-administration-workflows-status-is-slow

Cireson portal not loading after updated certificate

October 24, 2022 Posted by Alexander Axberg

If the portal is using a HTTPS binding, and you need to update that certificate, you also need to update the PlatformCache service since it sets the certificate in the binding in IIS.
If PlatformCache is not updated, it will automatically wipe the link to the certificate in the binding.

  1. Update the certificate in windows, in the computer cert. store.
  2. Copy the thumbprint of the newly installed certificate.
  3. Open the file Ciresonportal\Platform\Platform_cache.config and paste the thumbprint at: SslCertificateThumbprint
  4. Restart the Platformcache service.

Update TLS setting in Windows for Exchange Connector

March 18, 2022 Posted by Alexander Axberg

Microsoft has started to reject connections to ExchangeOnline webaccess unless you use at least TLS 1.2, during this week. This can cause your Exchange Connector to stop connecting to it’s mailbox, if you use M365.

To correct this, you need to add some registry keys to make .NET and Windows to use a higher TLS version as default.

More about this on these posts:

https://tdemeul.bunnybesties.org/2021/06/troubleshooting-scsm-exchange-connector.html

https://support.microsoft.com/en-us/topic/tls-1-2-protocol-support-deployment-guide-for-system-center-2016-58f3daa8-655e-e9e7-dafe-cbbd28203118

SQL deadlock errors in Orchestrator

January 20, 2022 Posted by Alexander Axberg

If you have many runbook instances that complete at the same time, you might end up with a lot of deadlock errors in the runbookservice log.
These deadlock can also make the runbook jobs fail with the error: Interrupted by user or runbook server

This can be caused by a missing index on the POLICY_REQUEST_ACTION_SERVERS table.
This is described in: https://docs.microsoft.com/en-us/troubleshoot/system-center/orchestrator/runbook-server-goes-offline

The KB above referes to Orchestrator 2012, but I’ve noticed that the index is missing in a brand new 2019 installation aswell.

Add the index with the following T-SQL:
CREATE NONCLUSTERED INDEX [IX_POLICY_REQUEST_ACTION_SERVERS_SeqNumber] ON [dbo].[POLICY_REQUEST_ACTION_SERVERS] ( [SeqNumber] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]

Now you should end up with a total of two indexes on the table.

File has not been pre-compiled, and cannot be requested

January 17, 2021 Posted by Alexander Axberg

If your Cirson portal suddenly starts throwing these errors when you try to access it, try these actions:

Unblock files
In powershell, run this command to unblock the files in the bin folder:
Get-Childitem c:\inetpub\Cireson\bin | Unblock-File

Clear .NET framework tempfiles
Delete all files in root folder in:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files

Data Warehouse upgrade fails with _MPProcessorDeferred error

April 1, 2020 Posted by Alexander Axberg

I ran into this error when upgrading the Data Warehouse from 2016 to 2019, at the stage when the import of new Management Packs begins.
The installation fails with the error: An error occurred while executing a custom action:_MPProcessorDeferred Upgrade failed after making permanent changes to this management server. A disaster recovery procedure will be necessary to rebuild this machine before attempting upgrade again. Consult the User Guide to determine which action to take next.
and the rolls back the installation.

The problem:
After looking in the SCSMInstall.log file, the installation failed due to a mismatch of collations.
This was caused by 2 columns in a table the DWStagingAndConfig database somehow had changed collation, that differs from the rest of the database.

The solution:

  1. Run a query against the DWStagingAndConfig database to find which table that contains the columns with wrong collation with the following query:
    SELECT col.name, col.collation_name, OBJECT_NAME(object_id) as ‘table’ FROM sys.columns col order by col.collation_name desc
  2. Find the columns with the mismatched collation and reset it to the correct one with, and make sure you set the correct data type and length:
    ALTER TABLE <tablename> ALTER COLUMN <columnname> nvarchar(255) COLLATE <correct collation>
  3. If you have a view with wrong collation (begins with: MTV), then (after updating all tables) just execute a: ALTER To->New Query, without changing anything, and the collation will be updated.
  4. If the installation continues to fail during the stage where the new Management Packs are imported, it could be due to some of the installation files already got updated. When the MSI is trying to update the MP/MPBS in the installation folder for SCSM (c:\program files\Microsoft System Center\Service Manager\), the created date and modified date has to be the same. If a previous installation has replaced them, the modified date may be newer. In that case just delete those files from the installation folder before running the installer.

“Cannot insert duplicate key row” -error in Transform.Common

February 5, 2020 Posted by Alexander Axberg

I recently ran into the following error in the Transform-job in Data Warehouse:

An error countered while attempting to execute ETL Module:
 ETL process type: Transform
 Batch ID: 5601
 Module name: TransformIncidentDim
 Message: ErrorNumber=”2601″ Message=”Cannot insert duplicate key row in object ‘dbo.IncidentDim’ with unique index ‘UniqueIndex’.” Severity=”14″ State=”1″ ProcedureName=”TransformIncidentDimProc” LineNumber=”163″ Task=”Inserting into Dimension”

The error occures when the transform job was trying to instart a specific Incident into the dbo.IncidentDim table in the DWRepository database.
The source of the error was that the specific category in this incident, existed 2 times in the IncidentClassification table, but I guess the same thing could happen to the support groups or any other enum in the incident class.

Solution:

  1. Open the IncidentClassification table in the DWRepository database, and look for enums with duplicate rows (but with different classificationID) with this query:
    SELECT count(enumTypeId) as ‘Count’ ,enumTypeId FROM IncidentClassification GROUP BY enumTypeId
  2. Look for a row where the count number is greater than 1. When found, copy the enumTypeId value and run this query:
    SELECT * FROM IncidentClassification WHERE enumTypeID = ‘<enumtypeID value>’
  3. To decide which duplicate row to remove, run a query against DWDataMart to see which one of the rows are used.
    Run this query against DWDataMart for each IncidentClassificationID above:
    SELECT * FROM IncidentDimvw WHERE Classification_IncidentClassificationID = ‘<IncidentclassificationID value>’
  4. For the row/rows in #2 that are not used in DWDataMart, delete them from DWRepository by using each the of IncidentClassificationIDs so only one row remains:
    DELETE FROM IncidentClassification WHERE IncidentClassificationID = ‘<IncidentclassificationID value>’
  5. Rerun the Transform.Common job, and it should complete successfully.

System Center 2019 released

March 14, 2019 Posted by Anders Asp

System Center 2019 has just been released and is available for download on both MVLS and MSDN / Visual Studio. In general, it’s a small release, and from a SCSM perspective the official news in this version are:

  • Support to SQL 2017
  • Improvement in Active Directory Connector
  • Improved UI Responsiveness
  • Enable Service Logon

Read more here:
https://docs.microsoft.com/en-us/system-center/scsm/whats-new-in-sm?view=sc-sm-2019

Also, be sure to read all the Known Issues in this release here:
https://docs.microsoft.com/en-us/system-center/scsm/release-notes-sm?view=sc-sm-2019

I’d hold back from upgrading my environments for a while until we’ve seen how stable the release is.

Also, from now on the semi-annual release cycle have been scrapped. So for those of you who went down that route you will now have to go back to the long term release cycle (in other words, SCSM 2019).

Read more here (under “Changes to release cadence” near the end):
https://cloudblogs.microsoft.com/windowsserver/2019/03/07/coming-soon-microsoft-system-center-2019/

Crashing tasks in Service Manager 2016

October 3, 2017 Posted by Anders Asp

So, I helped a customer upgrade to Service Manager 2016 the other day and the upgrade itself went fine. However, when trying to use some of their custom tasked after the upgrade, the console crashed.

The task itself isn’t anything special – it’s just a task to open a specific URL in their browser and has been working for a long time. We tried some different things to try to figure out what caused this crash. Running the console elevated, wiping our local SCSM user cache, trying from another client with another user and so on, but nothing seemed to help… When looking a little closer into the event log (Operations Manager log on the client), we discovered these events though.

And this part of the error message gave us a clue on what the issue might be:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.Xml.XmlException: ‘=’ is an unexpected token. The expected token is ‘;’. Line 1, position 399.

For some reason, it seems like Service Manager 2016 doesn’t like the syntax we’re using for our tasks anymore. So let’s take a look at the task itself:

After some testing, we figured out that the &-character is causing the crash. Apparently, it is not converted into correct XML syntax when stored into the management pack as it used to in previous version of Service Manager. To avoid the crashing tasks, we simply had to do this conversion ourselves. So instead of writing & in our tasks, we had to replace it with &amp; After this change, the tasks were working fine again without crashing the console.

Note that this also apply to text written in other places in the custom task, such as the Description field.

Using a standalone SSAS or SSRS when installing SCSM 2016

September 24, 2017 Posted by Anders Asp

I ran into some issues when trying to install a brand new Service Manager 2016 environment at a customer the other day. They have been using Service Manager for a while and instead of doing an in-place upgrade of their 2012 R2 environment, they wanted to do a side-by-side upgrade/migration and created new VMs with Windows Server 2016 and SQL 2016. As this customer had experienced issues with SSAS (SQL Server Analysis Services) and the Service Manager cube processing in the past, they decided to put SSAS on a separate server in an effort to minimize any issues.

This is all fine and supported, but for some reason the installer didn’t actually discover SSAS on this server when trying to install the Data Warehouse. Instead the wizard just showed us something in line with “SQL server is not available”. Looking in the setup log (SCSMSetupWizard01.log in the Temp folder for the user running the setup), we found this error message:

01:41:29:Failed while trying to check remote AS: Object reference not set to an instance of an object. –    at Microsoft.SystemCenter.Essentials.SetupFramework.HelperClasses.SetupValidationHelpers.GetHighestMajorVersion(IEnumerable`1 versionList)
at Microsoft.SystemCenter.Essentials.SetupFramework.HelperClasses.SetupValidationHelpers.GetManagementScope(String hostName, String sqlInstance, ConnectionOptions connectionOption, String wmiNamespace, String managementScopeDesc)
at Microsoft.SystemCenter.Essentials.SetupFramework.HelperClasses.SetupValidationHelpers.PreReqRemoteMachineforAS(String hostName)
at Microsoft.SystemCenter.Essentials.SetupFramework.HelperClasses.SetupValidationHelpers.ValidateAS(String hostName).

After some digging, we discovered that in order for the installer to actually identify SQL on the SSAS machine, we had to install the SQL Engine. After adding that to the SSAS machine the installation went fine, and you can actually go ahead and remove the SQL Engine when the installation is completed. The same fix is needed if you’re running SSRS on a stand-alone machine.