Skip to content

GitLab

Notice to GitKraken users: A vulnerability has been found in the SSH key generation of GitKraken versions 7.6.0 to 8.0.0 (https://www.gitkraken.com/blog/weak-ssh-key-fix). If you use GitKraken and have generated a SSH key using one of these versions, please remove it both from your local workstation and from your LRZ GitLab profile.

21.10.2021, 9:00 - 11:00: Due to updates GitLab may be unavailable for some minutes between 09:00 and 11:00.

Open
Created by Duo Li@ru58zid💬Developer

Job failed with nodes=8

The same job works with nodes=2 on NG but failed with more nodes e.g. nodes=8. OMP=8 and --ntasks-per-node=6

60.199790 [i01r01c04s06.sng.lrz.de],rank:40, core:16, tid:6 error exahype::solvers::ADERDGSolver::surfaceIntegral(...) Riemann solve was not performed on all faces of cell= (solverNumber:0,neighbourMergePerformed:[^A,^A,^A,^A,^@,^A],parentIndex:-1,type:Leaf,parentType:undefined,level:4,offset:[4,14,14],size:[1,1,1],previousTimeStamp:0,previousTimeStepSize:0.002805,timeStepSize:0.002805,timeStamp:0.002805,solutionIndex:288,solutionAveragesIndex:291,solutionCompressedIndex:-1,solution:0x7f2c50061600,solutionAverages:0x7f2c500d0e00,solutionCompressed:0,previousSolutionIndex:289,previousSolutionAveragesIndex:290,previousSolutionCompressedIndex:-1,previousSolution:0x7f2c50060100,previousSolutionAverages:0x7f2c500d0f00,previousSolutionCompressed:0,updateIndex:292,updateAveragesIndex:293,updateCompressedIndex:-1,update:0x7f2c50066080,updateAverages:0x7f2c500d7380,updateCompressed:0,extrapolatedPredictorIndex:294,extrapolatedPredictorAveragesIndex:295,extrapolatedPredictorCompressedIndex:-1,extrapolatedPredictor:0x7f2c4f9b7300,extrapolatedPredictorAverages:0x7f2c5008d800,extrapolatedPredictorCompressed:0,extrapolatedPredictorGradientIndex:-1,extrapolatedPredictorGradient:0,fluctuationIndex:296,fluctuationAveragesIndex:297,fluctuationCompressedIndex:-1,fluctuation:0x7f2c4f9bbb00,fluctuationAverages:0x7f2c5006b400,fluctuationCompressed:0,solutionMinIndex:298,solutionMaxIndex:299,solutionMin:0x7f2c50081c00,solutionMax:0x7f2c50081a00,facewiseAugmentationStatus:[2,2,2,1,3,1],augmentationStatus:2,facewiseCommunicationStatus:[2,2,2,2,1,2],communicationStatus:2,facewiseRefinementStatus:[0,0,0,0,0,0],refinementStatus:0,previousRefinementStatus:0,refinementFlag:0,compressionState:Uncompressed,bytesPerDoFInPreviousSolution:-1,bytesPerDoFInSolution:-1,bytesPerDoFInUpdate:-1,bytesPerDoFInExtrapolatedPredictor:-1,bytesPerDoFInFluctuation:-1,creation:UniformRefinement) (file:/hppfs/work/pr63qo/di52lak2/ExaHyPE_deadlock/ExaHyPE-Engine/./ExaHyPE/exahype/solvers/ADERDGSolver.cpp,line:1188)