High Performance Computing

Autodesk® CFD High Performance Computing (HPC) supports running simulations on multi-core computers as well as with clusters of computers.

A multi-core implementation is a computer that contains more than four cores. Autodesk® CFD sends the analysis data to CFDServer, which then sends the data to CFDSolver (the program that the performs the calculations) to be solved. The analysis is broken into multiple pieces and solved using the number of computer cores (CPUs) that are available.

A cluster is a group of computers connected together using a high speed network, typically INIFIBAND (a type of Ethernet). The cluster runs a version of windows called Microsoft High Performance Computing Server. Cluster computing uses multiple computers and all of their CPU's to solve the analysis.

Please check the Autodesk® CFD Knowledge Base for detailed instructions on how to implement HPC.

Terminology

• HPC: High Performance Computing
• Core: A single processor
• Head node: The primary server in a cluster environment
• Compute Nodes: Additional servers in a cluster environment

Requirements

Before you implement Autodesk® CFD HPC, you need to have a Cluster computing environment configured with the following elements:

• Windows HPC Server 2008 R2 or Windows Server 2012.
• Cluster Manager
• Configured Head and Compute Nodes

Implement HPC for Autodesk® CFD

Most of the process for configuring HPC for is automated, but there are a few important steps that must be completed:

1. Confirm that .net 4 and the Visual Studio 2010 Runtime environment are installed on the Head Node and all Compute Nodes.

2. Create a user account with admin privileges that is available on the Head Node and all Compute Nodes.

3. Install Autodesk® CFD on the Head Node.

4. Run HPCConfig.exe on the Head Node as the Admin account created in Step 2. (This file is located in the CFD installation directory.)

   HPCConfig performs the following tasks:

   • Creates and shares the network share directory
   • Restarts the SimCFdServ service under the login user
   • Tests the installation

5. Configure the Solver Computers dialog box only on the Head Node to include the accessible Compute Nodes:

   1. Click Setup (tab) > Simulation (panel) > Solver Computer (in the drop-down menu).
   2. To add a Compute Node, click ***Click to add a computer***, and enter the Computer Name.
   3. For each Compute Node, enter the number of accessible cores in the Cores column. Check the box in the Use Cluster column.
   4. Click OK when finished.

Registry Update

In some instances, the test model will fail. This is because an additional registry entry is needed on the Compute Nodes. A batch file is available which automates this process. To use this batch file, perform the following procedure:

1. From our website, download HPC_RegistrySetup.zip and extract the batch file HPC_RegistrySetup.bat to the Head Node.

2. Run the batch file, and note the location of the saved ".reg" file. This file contains the needed registry key.

   Note: This batch file creates a ".reg" file that contains this key: \HKEY_LOCAL_MACHINE\SOFTWARE\Autodesk\CFD\16.0

3. Copy this ".reg" file onto each Compute Node. To import the key into a Compute Node, login in as the Admin account. Double click the ".reg" file to run it.

Manual Procedure

You can export the registry entry from the Head Node and import into each Compute Node's registry manually:

To export the branch from the Head Node:

1. Open the Registry Editor. (Open a Command prompt, and type "regedit".)
2. Locate this key in the browser: \HKEY_LOCAL_MACHINE\SOFTWARE\Autodesk\CFD\17.0
3. Right click on the key, and click Export. Browse to a destination folder, and enter a name for the ".reg" file.
4. Copy the saved ".reg" file onto each Compute Node.

To import the key into a Compute Node, login in as the Admin account. Double click the ".reg" file to run it.

Firewall Exceptions

In some instances, you need to add firewall exceptions to the Head node as well as the Compute nodes. All of the exceptions should use a UNC path and point to the exe's within the Head node's shared directory. Typical exceptions include:

• \\headnode\CFD2017\CFDsolver.exe
• \\headnode\CFD2017\CFDserver.exe
• \\headnode\CFD2017\CFDJobManager.exe

Additional Configurations

You can configure CFD to run multiple jobs simultaneously on the cluster. To control the number of parallel jobs, download server_config.zip from our website and extract the configuration file,server_config.xml, to the Analyze directory on the Head Node. Each parallel job requires its own CFD solver license. For example, if you have 2 licenses you can run two jobs on the cluster concurrently.

To alter the number of parallel jobs, open the server_config.xml in Notepad and edit the following value:

• maxNumberOfHpcJobs - Maximum number of jobs you can run concurrently on a cluster (value cannot exceed number of available CFD licenses).

Recommended Hardware

Computing hardware is evolving very quickly, making it difficult to provide specs for the latest recommended hardware. The following hardware configurations were high end when this was written, but may have since been replaced. If in doubt, contact our Technical Support Team for hardware advice.

Hardware configuration for an HPC Cluster:

• Dual Quad Core Intel Xeon Processors (55XX or 56XX series)
• SAS (15K RPM) HDD
• At least 1 GB on-board RAM Graphics card
• 24GB of RAM on the head node and 12GB of RAM on the compute nodes.
• Infiniband cables to connect HPC systems

Hardware configuration for a multi-core machine:

• Dual Quad core Intel Xeon Processors (55XX series)
• SAS (15K RPM) HDD
• At least 1 GB on-board RAM Graphics card
• 12GB to 24GB of RAM, depending on average model size