Currently job artifacts in CI/CD pipelines on LRZ GitLab never expire. Starting from Wed 26.1.2022 the default expiration time will be 30 days (GitLab default). Currently existing artifacts in already completed jobs will not be affected by the change. The latest artifacts for all jobs in the latest successful pipelines will be kept. More information: https://gitlab.lrz.de/help/user/admin_area/settings/continuous_integration.html#default-artifacts-expiration

Commit dd2e6d59 authored by Timy Phan's avatar Timy Phan
Browse files

improved code quality

parents ce91257f d2890e8c
das
\ No newline at end of file
......@@ -10,14 +10,16 @@ __pycache__/
*.py[cod]
*$py.class
**/venv/
Tools/VadereAnalysisTools/VadereAnalysisTool/VadereAnalysisTool.egg-info/
Tools/VadereAnalysisTools/VadereAnalysisTool/vadereanalysistool.egg-info/
Tools/VadereAnalysisTools/VadereAnalysisTool/build/
Tools/VadereAnalysisTools/VadereAnalysisTool/dist/
Tools/VadereAnalysisTools/VadereAnalysisTool/vadereanalysistool**/output
Tools/ContinuousIntegration/**/output
# C extensions
*.so
vadere_analysis_tool/build
vadereanalysistool/build
#Tools and build utils
bin/
......@@ -41,6 +43,9 @@ VadereUtils/testreports/*.txt
VadereSimulator/resources/current_commit_hash.txt
VadereSimulator/testResources/**/*.legacy
#Java runtime files generated with "jdeps" and "jlink"
java-runtime/*
#model test output
VadereModelTests/**/output
VadereModelTests/*_private
......
......@@ -27,12 +27,14 @@ variables:
VADERE_PACKAGE_NAME_RELEASES: "vadere.${CI_COMMIT_TAG}.${CI_RUNNER_TAGS}.zip"
# Stage Definitions
# Watch out: the seed_test is after deploy, because currently it fails
# only sometimes and does not define a successful/unsuccessful state.
# Watch out: integration tests and the seed tests run after deployment, because
# currently tests fail very often because of timeouts. It is not clear if
# Vadere has slowed down somehow or the test environment is not reliable
# (i.e.,the virtual machines). We have to track that!
stages:
- unit_test
- integration_test
- deploy
- integration_test
- seed_test
# PLEASE, OMIT FOLLOWING GITLAB PITFALLS:
......@@ -55,7 +57,7 @@ stages:
artifacts:
when: on_success
paths:
- "*/target/site/coverage-reports"
- "*/target/coverage-reports"
expire_in: 1 week
when: on_success
......@@ -70,14 +72,14 @@ stages:
paths:
- "log_dir"
expire_in: 1 week
when: manual
when: on_success
.template_deploy_master_automatically:
stage: deploy
script:
- mvn clean
- mvn -Dmaven.test.skip=true package
- python3 -m zipfile -c ${VADERE_PACKAGE_NAME_BRANCHES} README.md VadereModelTests/ VadereGui/target/vadere.jar VadereSimulator/target/vadere-console.jar
- python3 -m zipfile -c ${VADERE_PACKAGE_NAME_BRANCHES} README.md VadereModelTests/ VadereGui/target/vadere-gui.jar VadereSimulator/target/vadere-console.jar
- scp ${VADERE_PACKAGE_NAME_BRANCHES} di49mur@webdev-mwn.lrz.de:~/webserver/htdocs/builds/master/${VADERE_PACKAGE_NAME_BRANCHES}
only:
refs:
......@@ -89,7 +91,7 @@ stages:
script:
- mvn clean
- mvn -Dmaven.test.skip=true package
- python3 -m zipfile -c ${VADERE_PACKAGE_NAME_RELEASES} README.md VadereModelTests/ VadereGui/target/vadere.jar VadereSimulator/target/vadere-console.jar
- python3 -m zipfile -c ${VADERE_PACKAGE_NAME_RELEASES} README.md VadereModelTests/ VadereGui/target/vadere-gui.jar VadereSimulator/target/vadere-console.jar
- scp ${VADERE_PACKAGE_NAME_RELEASES} di49mur@webdev-mwn.lrz.de:~/webserver/htdocs/builds/stable/${VADERE_PACKAGE_NAME_RELEASES}
# Run job only for "tags" (usually on the "stable" branch). Watch out: Multiple "only" tags are ORed together.
# See: https://gitlab.com/gitlab-org/gitlab-ce/issues/27818
......@@ -102,7 +104,7 @@ stages:
script:
- mvn clean
- mvn -Dmaven.test.skip=true package
- python3 -m zipfile -c ${VADERE_PACKAGE_NAME_BRANCHES} README.md VadereModelTests/ VadereGui/target/vadere.jar VadereSimulator/target/vadere-console.jar
- python3 -m zipfile -c ${VADERE_PACKAGE_NAME_BRANCHES} README.md VadereModelTests/ VadereGui/target/vadere-gui.jar VadereSimulator/target/vadere-console.jar
- scp ${VADERE_PACKAGE_NAME_BRANCHES} di49mur@webdev-mwn.lrz.de:~/webserver/htdocs/builds/branches/${VADERE_PACKAGE_NAME_BRANCHES}
when: manual
......@@ -111,7 +113,8 @@ stages:
script:
- mvn clean
- mvn -Dmaven.test.skip=true package
- python3 Tools/VadereAnalysisTools/VadereAnalysisTool/setup.py install --user
# setup.py install should from the directoty at which the setup.py file is located (working directory). The brackets return the working direcory back to previous position.
- (cd Tools/VadereAnalysisTools/VadereAnalysisTool/ && python3 setup.py install --user)
- python3 Tools/ContinuousIntegration/run_seed_comparison_test.py
artifacts:
when: on_failure
......
......@@ -10,7 +10,7 @@
* The new scenario element can be selected in `TopographyCreator` via the "emergency exit" button.
* The new scenario element is also taken into account by `SettingsDialaog` and `TikzGenerator`.
- Configured 1-click-deployment in ".gitlab-ci.yml".
* In short: Vadere is packaged as a single ZIP file containing following build artifacts: README.md VadereModelTests/ VadereGui/target/vadere.jar VadereSimulator/target/vadere-console.jar
* In short: Vadere is packaged as a single ZIP file containing following build artifacts: README.md VadereModelTests/ VadereGui/target/vadere-gui.jar VadereSimulator/target/vadere-console.jar
* Vadere is packaged for GNU/Linux and Microsoft Windows.
* The ZIP file can be accessed on the website via: http://www.vadere.org/releases/
* The website's filesystem contains three subdirectories for releases:
......@@ -27,7 +27,6 @@
- Logging API did *not* change. Each log level has an additional String formater
method like `log.errorf("Erron in %s with value of x: %d","ClassA", 42)`
- issues closed: [issue#163](https://gitlab.lrz.de/vadere/vadere/issues/163)
- Added script "osm2vadere.py" to "Tools" folder, to convert OpenStreetMap maps to a Vadere topography. (929d4775: Tools).
- `VadereConsole`: Add option `--logname <filename>` to specify the name for the log file.
Please note that the log file `log.out` is always written (because this file is configured
......@@ -64,10 +63,13 @@
### Changed
- Renamed Vadere JAR files.
* vadere.jar -> vadere-gui.jar
* postvis.jar -> vadere-postvis.jar
* vadere-console.jar -> remained unchanged
- Migration to Java 11 (OpenJDK).
- Removed directory `Documentation/version-control` which contained the Git hooks. The Git hooks are not required anymore. Instead, added `git rev-parse HEAD` to file `VadereSimulator/pom.xml` to create `VadereSimulator/resources/current_commit_hash.txt` during each build via `mvn compile`.
**Note:** The file `current_commit_hash.txt` is created during Maven's validation phase, i.e., before the actual build.
`
**Note:** The file `current_commit_hash.txt` is created during Maven's validation phase, i.e., before the actual build.
## v0.6 (2018-09-07)
......
The java_migration branch has used the following steps to compile and run vadere with JDK-11 on Windows 10:
0. Ensure that JDK-11 and Git is set in the environment variables
1. Adjust the IDE (IntelliJ)
1.1 Select File -> Settings -> Build, Execution, Deployment -> Compiler -> Java Compiler
1.1.1 Select 11 for "Project bytecode version"
1.1.2 Select 1.11 in "Per-module bytecode version" for each module
1.2 Select File -> Project Structure -> Project Settings -> Modules
1.2.1 Select "11 ..." for "Language level" for each module
2. Adjust pom.xml
2.1 search for <properties>
2.1.1 Change <maven-compiler.[source|target].version> to 1.11
2.2 Insert:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.0.0-M1</version>
<executions>
<execution>
<id>enforce-maven</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<requireMavenVersion>
<version>3.0</version>
</requireMavenVersion>
</rules>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<release>10</release>
</configuration>
</plugin>
</plugins>
</build>
3. Compile
3.1 Run "mvn -e clean test-compile compile" (-e for error stack trace)
3.* possibly required updates: "mvn versions:display-dependency-updates" and "mvn versions:display-plugin-updates"
4. Create local runtime
4.1 Run "jdeps --list-deps VadereGui/target/vadere.jar"
4.2 Run "jlink --no-header-files --no-man-pages --compress=2 --strip-debug --add-modules java.base,java.compiler,java.datatransfer,java.desktop,java.logging,java.management,java.naming,java.prefs,java.rmi,java.scripting,java.sql,java.xml --output java-runtime"
// The main output of jdeps is used as the parameter for jlink --add-modules.
4.3 Run "java-runtime/bin/java -jar VadereGui/target/vadere.jar"
\ No newline at end of file
......@@ -34,17 +34,17 @@ Stable releases and selected branch-builds are located on [www.vadere.org/releas
The ZIP file contains:
* **README.md** - this README file.
* **vadere.jar** - provides Vadere with a GUI.
* **vadere_console.jar** - provides the command line version of Vadere and allows easy integration into other applications.
* **vadere-gui.jar** - provides the GUI version of Vadere.
* **vadere-console.jar** - provides the command line version of Vadere and allows easy integration into other applications.
* **VadereModelTests** - contains test scenarios for pedestrian locomotion models. Note: The tests are also useful for a "getting started" (see below "Run Built-In Examples" for details).
To execute the `.jar` files it is highly recommended to use Oracle Java 8.
To execute the `.jar` files it is highly recommended to use Java 11 (OpenJDK).
### Build from Source
#### Dependencies
* Java 8
* Java 11 (OpenJDK recommended)
* Maven 3.0
* Git
* OpenCL (optional but recommended)
......@@ -56,10 +56,11 @@ To execute the `.jar` files it is highly recommended to use Oracle Java 8.
Vadere uses computer's video card to speed up some computations. Therefore, following OpenCL components should be installed:
* the OpenCL Installable Client Driver loader also called ICD loader (Opencl.dll for Windows and libOpenCL.so for Linux)
* OpenCL Drivers for your device
* drivers for your device(s)
Both should be offered by the vendor of your device (often there are also open-source solutions). The device can be a CPU as well as a GPU (recommanded). For example, if you have a NVIDIA GPU instaling your drivers should be enough to install both components.
Vadere will search for the best suiable device which is supported. On a desktop workstation this should be your video card (GPU). If there is no device supporting OpenCL Vadere will use a plain and slower Java-implementation instead.
Both should be offered by the vendor of your device. The device can be a CPU as well as a GPU (recommanded). For example if you have a NVIDIA GPU updating your drivers should be enough to install both components.
Vadere will search for the best device which is supported. On a desktop workstation this should be your video card (GPU). If there is no device Vadere will use a plain and slower Java-Implementation instead.
Please, use following instructions to set up the OpenCL components for your operating system:
* Windows: For further information using OpenCL on Windows read the paragraph Running an OpenCL application [click here](https://streamcomputing.eu/blog/2015-03-16/how-to-install-opencl-on-windows/).
......
# Extract line and branch coverage (in percentage) from HTML coverage reports
# which are created by Maven's jacoco plugin.
# which are created by Maven's JaCoCo plugin.
# Use top-level pom.xml to search in correct subdirectories.
#
# Wach out: call this script from root directory of project. E.g.
......@@ -11,6 +11,10 @@ import xml.etree.ElementTree as ET
import os
import re
# Use Unix path separators because this script is usually run in a Bash-context.
# And Bash fails with Windows path separators which are introduced by os.path.join(...)
path_separator = "/"
def get_modules_from_pom_file(filename="pom.xml"):
"""Return a list of submodules which where found in passed "pom.xml"."""
......@@ -33,15 +37,23 @@ def extract_line_and_branch_coverage(module_names):
module_to_coverage = dict()
default_coverage_file = "target/site/coverage-reports/index.html"
default_coverage_file = path_separator.join(["target", "coverage-reports", "index.html"])
for module in module_names:
coverage_path = os.path.join(module, default_coverage_file)
coverage_path = path_separator.join([module, default_coverage_file])
with open(coverage_path, "r") as file:
coverage_report = file.read()
# TODO: Regex seems be be broken on Windows CI worker. Find out why!
# This is a somewhat dirty fix for windows OS (I leave the todo open, if someone knows a cleaner solution.
# When parsing the html, there appear unicode(?) issues and in this case whitespaces are represented as Â\xa0
# I think this has something to do with it:
# https://stackoverflow.com/questions/13865346/why-would-a-python-regex-compile-on-linux-but-not-windows
coverage_report = coverage_report.replace(\xa0', ' ')
regex_pattern = re.compile(r"Total.*?([0-9]{1,3})\s?%.*?([0-9]{1,3})\s?%")
regex_pattern = re.compile(r"Total.*?([0-9]{1,3})%.*?([0-9]{1,3})%")
match = regex_pattern.search(coverage_report)
if match:
......@@ -49,6 +61,8 @@ def extract_line_and_branch_coverage(module_names):
branch_coverage = float(match.group(2))
module_to_coverage[module] = (line_coverage, branch_coverage)
else:
print(coverage_report)
print(f"match={match}")
raise Exception("Coverage data not found for module: {}".format(module))
return module_to_coverage
......
......@@ -3,8 +3,8 @@ import shutil
import subprocess
import time
from vadere_analysis_tool import VadereProject
from vadere_analysis_tool.analysis import OriginDeviation
from vadereanalysistool import VadereProject
from vadereanalysistool.analysis import OriginDeviation
def run_scenario_files_with_vadere_console(scenario_files, vadere_console="VadereSimulator/target/vadere-console.jar",
......
{
"name" : "complex_UNIT_001",
"description" : "",
"release" : "0.6",
"processWriters" : {
"files" : [ {
"type" : "org.vadere.simulator.projects.dataprocessing.outputfile.TimestepPedestrianIdOutputFile",
"filename" : "postvis.trajectories",
"processors" : [ 1, 2 ]
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.outputfile.TimestepPedestrianIdOverlapOutputFile",
"filename" : "overlaps.csv",
"processors" : [ 3 ]
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.outputfile.NoDataKeyOutputFile",
"filename" : "overlapCount.txt",
"processors" : [ 4 ]
} ],
"processors" : [ {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.PedestrianPositionProcessor",
"id" : 1
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.PedestrianTargetIdProcessor",
"id" : 2
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.PedestrianOverlapProcessor",
"id" : 3
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.NumberOverlapsProcessor",
"id" : 4,
"attributesType" : "org.vadere.state.attributes.processor.AttributesNumberOverlapsProcessor",
"attributes" : {
"pedestrianOverlapProcessorId" : 3
}
} ],
"isTimestamped" : true
},
"scenario" : {
"mainModel" : "org.vadere.simulator.models.osm.OptimalStepsModel",
"attributesModel" : {
"org.vadere.state.attributes.models.AttributesOSM" : {
"stepCircleResolution" : 18,
"numberOfCircles" : 1,
"varyStepDirection" : true,
"stepLengthIntercept" : 0.4625,
"stepLengthSlopeSpeed" : 0.2345,
"stepLengthSD" : 0.036,
"movementThreshold" : 0.0,
"optimizationType" : "NELDER_MEAD",
"movementType" : "ARBITRARY",
"dynamicStepLength" : true,
"updateType" : "EVENT_DRIVEN",
"seeSmallWalls" : false,
"minimumStepLength" : false,
"targetPotentialModel" : "org.vadere.simulator.models.potential.fields.PotentialFieldTargetGrid",
"pedestrianPotentialModel" : "org.vadere.simulator.models.potential.PotentialFieldPedestrianCompactSoftshell",
"obstaclePotentialModel" : "org.vadere.simulator.models.potential.PotentialFieldObstacleCompactSoftshell",
"submodels" : [ ]
},
"org.vadere.state.attributes.models.AttributesPotentialCompactSoftshell" : {
"pedPotentialIntimateSpaceWidth" : 0.45,
"pedPotentialPersonalSpaceWidth" : 1.2,
"pedPotentialHeight" : 50.0,
"obstPotentialWidth" : 0.8,
"obstPotentialHeight" : 6.0,
"intimateSpaceFactor" : 1.2,
"personalSpacePower" : 1,
"intimateSpacePower" : 1
},
"org.vadere.state.attributes.models.AttributesFloorField" : {
"createMethod" : "HIGH_ACCURACY_FAST_MARCHING",
"potentialFieldResolution" : 0.1,
"obstacleGridPenalty" : 0.1,
"targetAttractionStrength" : 1.0,
"timeCostAttributes" : {
"standardDeviation" : 0.7,
"type" : "UNIT",
"obstacleDensityWeight" : 3.5,
"pedestrianSameTargetDensityWeight" : 3.5,
"pedestrianOtherTargetDensityWeight" : 3.5,
"pedestrianWeight" : 3.5,
"queueWidthLoading" : 1.0,
"pedestrianDynamicWeight" : 6.0,
"loadingType" : "CONSTANT"
}
}
},
"attributesSimulation" : {
"finishTime" : 90.0,
"simTimeStepLength" : 0.4,
"realTimeSimTimeRatio" : 0.0,
"writeSimulationData" : true,
"visualizationEnabled" : true,
"printFPS" : false,
"digitsPerCoordinate" : 2,
"useFixedSeed" : true,
"fixedSeed" : -3213925745664992646,
"simulationSeed" : 0
},
"topography" : {
"attributes" : {
"bounds" : {
"x" : 0.0,
"y" : 0.0,
"width" : 50.0,
"height" : 50.0
},
"boundingBoxWidth" : 0.5,
"bounded" : true
},
"obstacles" : [ {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 0.4,
"y" : 0.4
}, {
"x" : 0.5,
"y" : 18.7
}, {
"x" : 9.8,
"y" : 18.8
}, {
"x" : 15.4,
"y" : 8.3
}, {
"x" : 15.4,
"y" : 0.5
} ]
},
"id" : 9
}, {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 0.6,
"y" : 34.9
}, {
"x" : 0.4,
"y" : 49.5
}, {
"x" : 14.9,
"y" : 49.5
}, {
"x" : 14.7,
"y" : 42.3
}, {
"x" : 9.0,
"y" : 35.0
} ]
},
"id" : 10
}, {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 36.0,
"y" : 25.7
}, {
"x" : 12.4,
"y" : 25.8
}, {
"x" : 12.5,
"y" : 29.5
}, {
"x" : 20.7,
"y" : 39.6
}, {
"x" : 32.0,
"y" : 39.5
}, {
"x" : 36.1,
"y" : 32.7
} ]
},
"id" : 11
}, {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 19.8,
"y" : 22.1
}, {
"x" : 32.3,
"y" : 22.2
}, {
"x" : 33.24321348467342,
"y" : 18.125317746210836
}, {
"x" : 43.4,
"y" : 18.2
}, {
"x" : 43.5,
"y" : 3.9
}, {
"x" : 35.9,
"y" : 3.9
}, {
"x" : 29.8,
"y" : 11.3
}, {
"x" : 29.800000000000008,
"y" : 11.311504424778763
}, {
"x" : 19.8,
"y" : 11.4
} ]
},
"id" : 12
}, {
"shape" : {
"x" : 35.5,
"y" : 39.3,
"width" : 14.3,
"height" : 10.2,
"type" : "RECTANGLE"
},
"id" : 5
} ],
"stairs" : [ {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 13.0,
"y" : 29.8
}, {
"x" : 7.9,
"y" : 33.7
}, {
"x" : 14.8,
"y" : 42.1
}, {
"x" : 20.3,
"y" : 38.8
} ]
},
"id" : 20,
"treadCount" : 35,
"upwardDirection" : {
"x" : 1.0,
"y" : 1.3
}
}, {
"shape" : {
"x" : 43.1,
"y" : 3.9,
"width" : 6.4,
"height" : 14.4,
"type" : "RECTANGLE"
},
"id" : 14,
"treadCount" : 45,
"upwardDirection" : {
"x" : 0.0,
"y" : 1.0
}
} ],
"targets" : [ {
"id" : 6,
"absorbing" : false,
"shape" : {
"x" : 41.6,
"y" : 20.0,
"width" : 3.7,
"height" : 10.0,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
}, {
"id" : 7,
"absorbing" : true,
"shape" : {
"x" : 19.9,
"y" : 41.7,
"width" : 10.0,
"height" : 3.7,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
}, {
"id" : 8,
"absorbing" : true,
"shape" : {
"x" : 20.0,
"y" : 4.6,
"width" : 10.0,
"height" : 3.0,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
}, {
"id" : 15,
"absorbing" : false,
"shape" : {
"x" : 5.2,
"y" : 20.0,
"width" : 3.5,
"height" : 9.8,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,