Scalability Tests (MPI,TBB,MPI+TBB)
This is my spreadsheet.
Tests
Testing scalability with parameters
- dim=2,3,
- p=3,5,7,9
- h=???
Systems
system | processor | cores per processor | cores per node | memory per node |
---|---|---|---|---|
phi1 | 2 Intel Xeon E5-2650 2.0 GHz | 8 cores per processor | 16 cores per node | 64 GByte RAM |
ADER-DG
Additional parameters:
- ts=fused,standard,
Runs;
-
TBB
- Strong scaling
- Weak scaling
-
MPI
- Strong scaling
- Weak scaling
-
MPI+TBB
- Strong scaling
- Weak scaling
Godunov FV
Additional parameters:
Comment: Fused time stepping results here in a single sweep scheme.
- ts=fused,standard
Runs;
-
TBB
- Strong scaling
- Weak scaling
-
MPI
- Strong scaling
- Weak scaling
-
MPI+TBB
- Strong scaling
- Weak scaling
Limiting ADER-DG (ADER-DG + FV)
Comment: Fused time stepping is not supported yet for the limiting ADER-DG solver.
Additional parameters:
- ts=standard,fused
Runs;
-
TBB
- Strong scaling
- Weak scaling
-
MPI
- Strong scaling
- Weak scaling
-
MPI+TBB
- Strong scaling
- Weak scaling
Setting up experiments on Hamilton (Durham's HPC)
Here I will document an (more or less) efficient approach to run the above experiments on Hamilton.
Installing the latest ExaHyPE and Peano snapshots
-
Installing ExaHyPE: Hamilton has git version 1.7.1 installed. Unfortunately, I had login issues while I was trying to clone the ExaHyPE-Engine reppository. I am thus required to perform a two stage process to get ExaHyPE on my login node.
- Check ExaHyPE out on my laptop.
- Optional: scp to mira. Login to mira.
- scp to hamilton.
-
Installing Peano: Hamilton has svn version 1.6.11 installed.
- Obtaining Peano is very convenient. It is done via:
svn checkout http://svn.code.sf.net/p/peano/code/trunk peano
- Obtaining Peano is very convenient. It is done via:
Building ExaHyPE
...
Job scripts
We have currently three different job script "frameworks" available in the ExaHyPE-Engine repository. While Dominics job scripts are only suited for TBB on Hamilton and SuperMUC, Tobias' job script suite is suited to perform MPI and MPI+TBB benchmarks on Hamilton (DUR). Vasco's python based job script framework can basically run anything on SuperMUC.
I will try to merge the three approaches into a python version with different backends based on the supercomputer.
-
Tobias' bash job scripts can be found at
~/dev/codes/c/ExaHyPE/ExaHyPE-Engine/Miscellaneous/JobScripts/hamilton
. -
Dominics' bash job scripts can be found at
~/dev/codes/c/ExaHyPE/ExaHyPE-Engine/Miscellaneous/JobScripts/scaling
. -
Vasco's python job scripts can be found at
~/dev/codes/c/ExaHyPE/ExaHyPE-Engine/Miscellaneous/JobScripts/
.
Performance analysis scripts
- Analysing domain decomposition: ...
- Plotting speedups: ...
Benchmark March 2017
- Dominic's MPI and MPI+TBB job scripts for Hamilton (SLURM) and SuperMUC (LoadLeveler) Benchmark_Mar2017.tar.gz