Improve memory efficiency of output processors
Problem to solve
Currently, the output processors keep all their data into memory and write everything to disk after the simulation is finished. For larger scenarios, this increases the memory immensely (which lowers the ability for parallel computations, e.g. with the suq-controller).
Generally, there are two types of processors:
- Only require information from the current state
- Require information from past states (e.g. speed average over the last 5 time steps, or a PCA over the entire data)
The second type is more tricky. In a discussion we agreed to handle the second case as follows:
- Write out the information to a file (**)
- After the simulation is done read the data again and transform it accordingly.
(**) Note: this should be asynchronously to not slow down the simulation too much (this part requires probably more discussion!)
List of processors that can write data in an online fashion:
- AreaDensityCountingProcessor
- AreaDensityVoronoiProcesor
- AreaSpeedProcessor
- BonnMotionTrajectoryProcessor (needs refactoring)
- FundamentalDiagramAProcessor
- FundamentalDiagramBProcessor
- FundamentalDiagramCProcessor
- FundamentalDiagramDProcessor
- FundamentalDiagramEProcessor
- GroupMemberEuclideanDistance
- GroupMemberPotentialDist
- GroupMemberSeparatedObstacle
- PedestrianDensityCountingProcessor
- PedestrianDensityGaussianProcessor
- PedestrianFlowProcessor
- PedestrianGroupIDProcessor
- PedestrianGroupMaxDistProcessor
- PedestrianGroupSizeProcessor
- PedestrianLineCrossProcessor
- PedestrianOffsetPositionProcessor
- PedestrianOSMStrideLengthProcessor
- PedestrianOverlapProcessor
- PedestrianPositionProcessor
- PedestrianSourceIdProcessor
- PedestrianStartTimeProcessor
- PedestrianStateProcessor
- PedestrianStateProcessor
- PedestrianTargetIdProcessor
- PedestrianVelocityDefaultProcessor
- PointDensityCountingAlgorithm
- PointDensityGaussianAlgorithm
- QueueWidthProcessor
- SumVoronoiAlgorithm
List of processors that require past data (offline, delayed online, preprocess required)
- EvacuationTimeProcessor
- MaxOverlapProcessor
- MeanPedestrianEvacuationTimeProcessor
- NumberOverlapsProcessor
- PedestrianLastPositionProcessor
- PedestrianTrajectoryProcessor
- PedestrianVelocityByTrajectoryProcessor
- PedestrianVelocityProcessor
Unknown
- PedestrianBehaviourProcessor
- PedestrianCrossingTimeProcessor
- PedestrianEndTimeProcessor
- PedestrianFootStepProcessor
- PedestrianWaitingEndTimeProcessor
- PedestrianWaitingTimeProcessor
- TargetFloorFieldGridProcessor
Edited by Schuhbaeck, Stefan