Write script to run performance measures of Vadere for every commit
Wishlist:
- Requires some representative scenarios to test for performance
- Check out the option "cache: Define list of files that should be cached between subsequent runs"
- Make plots and tables to download (as artifacts) that can be downloaded
- a table would be: "date", "commit", "scenario name", "wall clock time", "difference last"
- plots can be done for each scenario and aggregated over all tested scenarios (removing, changing and adding new scenarios has to be considered in the aggregating part -- somehow scenario changes have to be handled
- Handle "benchmark" runs with gitlab -> CI/CD -> Schedule a new pipeline, once per day
- Additionally, insert into .yaml file a job that only starts
when: manual
see: https://stackoverflow.com/questions/42986385/how-to-run-a-specific-job-in-gitlab-ci - Include the performance-measure script as a testing case in suqc, to make sure it is running (make in a separate folder "scripts")
- The current
suqc
should be installed on the worker - make a reinstall if current version does not match on gitlab-worker (or always reinstall...) - Enable the Java profiler for each scenario run and provide it in the artifacts (see https://stackoverflow.com/questions/11123727/is-there-a-console-profiler-for-java)
Edited by Ghost User