Digital design exploration opens the door to fast innovation at lower costs. And the best is with Simcenter STAR-CCM+ you´ll get the tool to do that within a familiar environment. Since 2017 STAR-CCM+ is equipped with the Design Manager that allows you to do automated design studies. In this week´s blog I would like to enlighten about how you can utilise the Design Manager on different resources.
Direct Submission
The most straight forward way to run the Design Manager is of course to run it, as well as the design evaluations, locally. Also referred as Direct submission. But resources that you need for some simulations are not always available right where you are.
So, you might consider running the simulation actually on a cluster or workstation. No additional installation or configuration is required to run Design Manager on a cluster. However, Design Manager requires that a Batch Management System, i.e. a queue system, is present on the cluster.
For cluster with a queue systems installed Design Manager offers the following modes in which to run Design Manager on a cluster:
- General Job Submission
- Pre-Allocation
- Job Manager
Design Manager without a queue system
Workstation
Smaller clusters which are accessed by only a few users or workstations often do not have a batch management system. For such systems it can be convenient to perform a design exploration in batch mode.
The steps to run the DM project in batch are as follows:
- Prepare the Design Manager project on the local machine. Create a new Study and specify the required data, e.g. parameters, responses/objectives or constraints.
- Edit the study setting for parallel computing.
- Modify the Simultaneous Jobs and Compute Processes properties under Design Studies->Design Study->Settings->Run Settings (see images below). Also make sure that the Compute Resource type is set to “Direct”.
Note that the Compute Processes value here is the number of cores per job, therefore the total number of cores required for the study would be “Simultaneous Jobs” x “Compute Processes”
3. Transfer the following files onto the workstation or cluster node where you want to run the simulations:
-
- sim file
- dmprj file
4. Execute the Design Manager project with the batch command.
starccm+ [project] -batch -dmnoshare -passtodesign "-power -podkey <value> -licpath 1999@flex.cd-adapco.com"
Where:
-dmnoshare
Specifies the Unlicensed Design Manager Server Licensing Scheme.
-passtodesign <options>
Specifies simulation options to pass to the design simulations, such as License Options or other Simcenter STAR-CCM+ options.
The design study will be run automatically. In case you are on the network as the resource, you can connect your local STAR client to the server as usual.
Multiple cluster nodes without queue system
In case you want to combine several cluster node resources, you can log in on one of the nodes and start the Design Manager from there in batch, as described above. However, in some cases shared file system across all nodes is set aside and the STAR instance cannot access the location of the variant. To bypass this, you can set up the Compute Resources to Linux Cluster, yet you only provide a script file that reassembles the path and provide the information for the computer nodes.
#!/bin/bash
#Read design information
STARCASE=$(cat dm_design_execute_options.txt) #Design name
CASELOG=$(cat dm_design_output_info.msg) #output log file
#Reassemble the working directory
WORKINGDIR=$(pwd)
read -a FLAG <<< "$STARCASE"
SIMFILE="$WORKINGDIR/${FLAG[2]}"
#Submit a simulation to computer nodes
starccm+ -batch ${FLAG[1]} -power -mpi platform -on compute-0-1.local:64,compute-0-2.local:64 $SIMFILE >> $CASELOG
exit 0 Design Manager with a queue system
Batch systems manage the allocation of computational jobs to a single large computer or a cluster of smaller computers.
General Job Submission
In general job submission mode, Design Manager submits each design simulation as a single job to the queue. For this mode, the Design Manager server must run on a node that has the ability to submit jobs to the queue—such as the cluster head node or a visualization node. This cluster node and the compute nodes must share the same file system with the same mount path. After starting the design study from the Design Manager client, Design Manager submits the design simulations to the queue, tracks their progress, and extracts their output on completion.
You specify the settings for the submission in Compute Resources > Linux Cluster. Here you can also provide the path to a submission script.
Pre-Allocation
The pre-allocation mode allows you to submit the entire Design Manager project as a single job to the cluster queue. Within the allocated resources, Design Manager acts as its own queueing system and submits the design simulations according to the settings in the study (that is, the number of simultaneous jobs and the number of cores per job). The resources that you allocate for the Design Manager project must be sized according to the resources desired for the design studies in the project.
Pre-Allocated resources can for instance be accessed in an interactive session. On Grid Engine you specify this would be
qlogin -pe mpi 64
Once your interactive session is established, you can start the Design Manager in batch mode with
starccm+ [project] -batch -dmnoshare
However, the Design Manager project needs to be set up and prepared before in an GUI session. The batch command will start the Design Study immediately. You also set the Compute Resource type to “Direct” again since the Design Manager is located directly where the simulations are run.
Job Manager
Job Manager feature aims to help the simulation engineer to easily and transparently submit Design Manager studies from the Simcenter STAR-CCM+ environment to compute resources without the need to understand them. It also aims to help the administrators (IT or power user) to easily manage resources and users’ rights.
Job Manager acts as an interface between your local machine and a set of resources. It permits you to access remote compute resources and control them without the need to log into them and run scripts directly and easily.
From the Compute Resources you select Job Manager and log in to it with your user name and password. Subsequently, you get access to the pre defined submission templates.
I hope this week’s read will be helpful to you get your design exploration done in an efficient way such that you can make smart decisions, faster! Do not hesitate to reach out at support@volupe.com, if you have any questions.
Read also:
Bubble Plots for Design Exploration
Directed meshing in Simcenter STAR-CCM+
Release of Simcenter STAR-CCM+ 2020.2 part 4
