HPC Pack 2019 Update 2 – High Availability Instructions

This document is created for highly available head nodes using SQL PaaS for the databases. The current documentation for this option starts in section “2.x”.

Upgrade to HPC Pack 2019 Update 2 | Microsoft Learn

  1. Stop all running jobs.
  2. From HPC Pack Cluster Manager, set all nodes to offline.
  3. Run the following PowerShell command to stop the Azure auto grow and shrink service if it is enabled.
  4. First gather the current settings as this will need to be re-enabled after the update

PowerShell to retrieve current settings for grow shrink options.

Add-PSSnapin Microsoft.Hpc

Get-HpcClusterProperty -AutogrowShrink

PowerShell change to disable the growing and shrinking during the udpate– make change to shrink to zero.

Add-PSSnapin Microsoft.Hpc

Set-HpcClusterProperty -EnableGrowShrink 0

  1. Optionally back up the HPC databases manually.
    1. (SQL PaaS is backing up constantly, but we can verify this setting
  2. Download HPCPack2019Update2-Patch-v7756.zip from Microsoft Download Center. Up-zip the package to your local folder of the head node(s).

Patch Link: https://download.microsoft.com/download/0/7/a/07a7ab5b-1ac4-4eaf-a360-00edc51a3229/HPCPack2019Update2-Patch-v7756.zip

** Note: on the patch zip file after download go to properties of the zip file and “unblock” it.

Upgrade built-in high availability head nodes

Run HPCPack2019Update2_x64.exe as administrator on each of the head nodes, and reboot if required.

It is recommended to run the following PowerShell cmdlet to find the primary head node and do the upgrade on secondary head nodes first.

PowerShell

Get-HpcServicePrimary <– this command will show the current primary head node. Start with the secondary.

Example screens during the installation of the HPCPack2019Update2_x64.exe.

A screenshot of a computer error

Description automatically generated
A screenshot of a computer software update

Description automatically generated
A screenshot of a computer error

Description automatically generated
A screenshot of a computer error

Description automatically generated
A screenshot of a computer error

Description automatically generated
  • Install on the next head node. Reboot if needed.

Replace the installation package in the install share for the new files in Update 2.

Unzip the InstallShare.zip file, and copy all the sub-folders and files to the folder %CCP_DATA%InstallShare and replace the files in the destination.

“C:\Program Files\Microsoft HPC Pack 2019\Data\InstallShare”

This just needs to be completed on 1 Headnode, the folder doesn’t exist on the second head node.

Upgrade Windows compute nodes, broker nodes, un-managed server nodes and workstation nodes

  1. Copy HPCPack2019Update2_x64.exe (or HPCPack2019Update2_x86.exe for workstation node with 32-bit operating system) to a network share folder (such as \\HPCHN01\HPCUpdates) which everyone has read permission
  2. In this example you could put the installer in this folder: \\hpchn01\REMINST\Updates\
  3. Open a command prompt console as administrator on one head node, and use the clusrun utility to upgrade the nodes, for example:

Windows Command Prompt

clusrun /nodegroup:ComputeNodes \\HPCHN01\REMINST\Updates\HPCPack2019Update2_x64.exe -unattend -SystemReboot

Example Output of clusrun command:

A black screen with white text

Description automatically generated

If workstation nodes are used run the following:

clusrun /nodegroup:WorkstationNodes \\HPCHN01\HPCUpdates\HPCPack2019Update2_x64.exe -unattend -SystemReboot

 Note

If the cluster administrator doesn’t have administrative privileges on workstation nodes and unmanaged server node, the clusrun utility may not be able to apply the update. In these cases the update should be performed by the administrator of the workstation and unmanaged servers.

On-premises Desktop machines that submit jobs:

Download the zip file and extract it to a local share that on-premises users have access to. In this example the remote on-premises hpc clients do not have SMB access to Azure file share of the head nodes. Downloading the files from the zip below and sharing locally on a file share is a good option.

https://download.microsoft.com/download/0/7/a/07a7ab5b-1ac4-4eaf-a360-00edc51a3229/HPCPack2019Update2-Patch-v7756.zip

Run the HPCPack2019Update2_x64.exe to upgrade the HPC pack client utilities.

HPCPack2019Update2_x64.exe will check to see if the HPC client is installed, and if it installed it will uninstall the older version and reinstall the new version. This version does not by default set the client certificate option to 1 like the prior HpcClient_x64.msi installation did.

A screenshot of a computer program

Description automatically generated

Add/remove programs for the HPC Client Components should show the version as 6.2.7756.0.

A close up of a logo

Description automatically generated

Post Install

After the installation verify the version number of the head nodes:

A screenshot of a computer error

Description automatically generated

Reset PowerShell properties to what they were previously.

PowerShell

Add-PSSnapin Microsoft.Hpc

Set-HpcClusterProperty -EnableGrowShrink 1

NOTES:

Remote client Notes:

On a new client machine that have never had the HPC Pack utilities installed, ensure that the client has C++ runtime installed:

https://aka.ms/vs/17/release/vc_redist.x64.exe

The Client is backwards compatible. In situations where the Dev/Test was updated, but production has not been yet the updated client should work on both environments.

Database Notes:

That Database that is updates is the HpcScheduler database with the following change. If a restore needed to be completed this database is what was updated.

ALTER TABLE JobProjectName ALTER COLUMN Name nvarchar(128) NULL

About the author

Jeff Pigott is a Senior Cloud Solution Architect at Microsoft, specializing in Azure Infrastructure technologies including Azure Arc, Azure Update Manager, SQL DB, security, and storage. With over 29 years of experience in IT, development, and DevOps, Jeff has worked with a wide range of platforms, tools, and languages. Jeff is passionate about staying up-to-date with the latest Microsoft technologies and services, and holds multiple Azure certifications.