Sisu Scalability Tests

CSC's supercomputer Sisu (sisu.csc.fi) is intended for well-scaling parallel applications. The minimum size of a parallel job is 72 compute cores and the maximum size is either 1008 compute cores, if a user has not demonstrated the parallel performance of their software, or at most 9600 compute cores, if they have done it. The purpose of this restriction is to ensure fair and efficient usage of Sisu's resources.

Performing scalability tests

CSC requires that the users wishing to run parallel jobs with more than 1008 compute cores  will evaluate and report the scalability of their software on Sisu. As a preparatory step of the scaling test process, run the scalability tests for your code up to 1008 cores (for example with 120, 240, 480 and 960 cores). Doing these small scale scaling tests is very useful as they will help users to utilize their computing quotas optimally

After the preparatory scaling, test contact the CSC Service Desk for more instructions. The largest number of cores that a user can employ is decided later on, based on the large scale scalability tests that will be performed in collaboration with CSC specialists.

Scalability criteria

The current parallel scalability criterion for Sisu is (this is subject to change):

  • When the number of cores is doubled, the wall time must decrease by a factor of 1.5.

The scalability is evaluated through tests. Here are some guidelines for performing the tests:

  • Scalability tests are run through the batch job system.
  • The input data set used for the test runs should reflect real production runs.
  • The input data set must be the same for each run.
  • Parameters affecting the scalability can, and are encouraged to be, changed.

If the parallel performance of a software is self-evident, based on earlier tests carried out on Sisu or other supercomputer, no further tests are required. Sending an announcement to servicedesk@csc.fi is sufficient in such a case. Note however, that even if a software is shown to scale well with a certain data set, not all jobs (e.g. smaller data sets) scale as well. In order to use the resources efficiently it is good to test scalability for each job type before extensive simulations.

Assistance

CSC's experts can help users in performing scalability tests if needed and also provide hints for improving the performance of their software. Contact CSC Service Desk if you need assistance with your software.

Reporting

The scalability report should contain a short description of the software and the test case, as well as wall times with each compute core count. Especially the factors that affect the scalability of the software should be elicited. The report is sent to servicedesk@csc.fi.


Examples of scaling test results

CP2K, 512-H2O results on Sisu

cores 1/s per step sec/step speedup
120 0.02125 47.0589 -
240 0.035383 28.2622 1.665083
480 0.054041 18.5044 1.527323
960 0.072745 13.7467 1.346098
1920 0.086688 11.5356

1.191676

--> maximum allowed number of cores is 480

NAMD apoa1 scaling results on Sisu (92k atoms)

cores perf [ns/day] speedup
120 12.82 -
240 24.14 1.88
480 41.06 1.70
960 66.10 1.61
1440 86.19 1.61
1920 95.45 1.44

--> maximum allowed number of cores is 1440 (but note, that Gromacs provides for a bigger system more ns/day with a lot less cores! The parameters might not be directly comparable, though.)

Gromacs d.dppc modified to use PME results on Sisu (121k atoms)

cores perf [ns/day] speedup
24 19.858  
48 37.681 1.90
72 52.22 1.82
96 67.196 1.78
192 106.194 1.58
288 142.046 1.64
384 186.694 1.76
480 172.449 1.21
720 219.004 1.22
768 121.725 0.65
--> maximum allowed number of cores is 384 (note the performance drop at 480 cores. This is typical for Gromacs. Performance might be better at some other core count close to that value and highlights the importance of testing scaling).