Skip to content

GitLab

Commit 9c8b9a99 authored by Daniel Lehmberg's avatar Daniel Lehmberg
Browse files

Merge branch 'java_migration' into 'master' 

Migration to OpenJDK 11, solves #218 #217 #195 #210 #209

Closes #209

See merge request !47
parents 26068c1e b558728f
Pipeline #94674 failed with stages
in 77 minutes and 56 seconds
Hide whitespace changes
Inline Side-by-side
.dummy deleted 100644 → 0
View file @ 26068c1e
das
\ No newline at end of file
.gitignore
View file @ 9c8b9a99
......@@ -10,14 +10,16 @@ __pycache__/
*.py[cod]
*$py.class
**/venv/
Tools/VadereAnalysisTools/VadereAnalysisTool/VadereAnalysisTool.egg-info/
Tools/VadereAnalysisTools/VadereAnalysisTool/vadereanalysistool.egg-info/
Tools/VadereAnalysisTools/VadereAnalysisTool/build/
Tools/VadereAnalysisTools/VadereAnalysisTool/dist/
Tools/VadereAnalysisTools/VadereAnalysisTool/vadereanalysistool**/output
Tools/ContinuousIntegration/**/output
# C extensions
*.so
vadere_analysis_tool/build
vadereanalysistool/build
#Tools and build utils
bin/
......@@ -41,6 +43,9 @@ VadereUtils/testreports/*.txt
VadereSimulator/resources/current_commit_hash.txt
VadereSimulator/testResources/**/*.legacy
#Java runtime files generated with "jdeps" and "jlink"
java-runtime/*
#model test output
VadereModelTests/**/output
VadereModelTests/*_private
......
.gitlab-ci.yml
View file @ 9c8b9a99
......@@ -27,12 +27,14 @@ variables:
VADERE_PACKAGE_NAME_RELEASES: "vadere.${CI_COMMIT_TAG}.${CI_RUNNER_TAGS}.zip"
# Stage Definitions
# Watch out: the seed_test is after deploy, because currently it fails
# only sometimes and does not define a successful/unsuccessful state.
# Watch out: integration tests and the seed tests run after deployment, because
# currently tests fail very often because of timeouts. It is not clear if
# Vadere has slowed down somehow or the test environment is not reliable
# (i.e.,the virtual machines). We have to track that!
stages:
- unit_test
- integration_test
- deploy
- integration_test
- seed_test
# PLEASE, OMIT FOLLOWING GITLAB PITFALLS:
......@@ -55,7 +57,7 @@ stages:
artifacts:
when: on_success
paths:
- "*/target/site/coverage-reports"
- "*/target/coverage-reports"
expire_in: 1 week
when: on_success
......@@ -70,7 +72,7 @@ stages:
paths:
- "log_dir"
expire_in: 1 week
when: manual
when: on_success
.template_deploy_master_automatically:
stage: deploy
......@@ -111,7 +113,8 @@ stages:
script:
- mvn clean
- mvn -Dmaven.test.skip=true package
- python3 Tools/VadereAnalysisTools/VadereAnalysisTool/setup.py install --user
# setup.py install should from the directoty at which the setup.py file is located (working directory). The brackets return the working direcory back to previous position.
- (cd Tools/VadereAnalysisTools/VadereAnalysisTool/ && python3 setup.py install --user)
- python3 Tools/ContinuousIntegration/run_seed_comparison_test.py
artifacts:
when: on_failure
......
Guideline for Java 11.txt 0 → 100644
View file @ 9c8b9a99
The java_migration branch has used the following steps to compile and run vadere with JDK-11 on Windows 10:
0. Ensure that JDK-11 and Git is set in the environment variables
1. Adjust the IDE (IntelliJ)
1.1 Select File -> Settings -> Build, Execution, Deployment -> Compiler -> Java Compiler
1.1.1 Select 11 for "Project bytecode version"
1.1.2 Select 1.11 in "Per-module bytecode version" for each module
1.2 Select File -> Project Structure -> Project Settings -> Modules
1.2.1 Select "11 ..." for "Language level" for each module
2. Adjust pom.xml
2.1 search for <properties>
2.1.1 Change <maven-compiler.[source|target].version> to 1.11
2.2 Insert:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.0.0-M1</version>
<executions>
<execution>
<id>enforce-maven</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<requireMavenVersion>
<version>3.0</version>
</requireMavenVersion>
</rules>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<release>10</release>
</configuration>
</plugin>
</plugins>
</build>
3. Compile
3.1 Run "mvn -e clean test-compile compile" (-e for error stack trace)
3.* possibly required updates: "mvn versions:display-dependency-updates" and "mvn versions:display-plugin-updates"
4. Create local runtime
4.1 Run "jdeps --list-deps VadereGui/target/vadere.jar"
4.2 Run "jlink --no-header-files --no-man-pages --compress=2 --strip-debug --add-modules java.base,java.compiler,java.datatransfer,java.desktop,java.logging,java.management,java.naming,java.prefs,java.rmi,java.scripting,java.sql,java.xml --output java-runtime"
// The main output of jdeps is used as the parameter for jlink --add-modules.
4.3 Run "java-runtime/bin/java -jar VadereGui/target/vadere.jar"
\ No newline at end of file
README.md
View file @ 9c8b9a99
......@@ -38,13 +38,13 @@ The ZIP file contains:
* **vadere_console.jar** - provides the command line version of Vadere and allows easy integration into other applications.
* **VadereModelTests** - contains test scenarios for pedestrian locomotion models. Note: The tests are also useful for a "getting started" (see below "Run Built-In Examples" for details).
To execute the `.jar` files it is highly recommended to use Oracle Java 8.
To execute the `.jar` files it is highly recommended to use Oracle Java 11.
### Build from Source
#### Dependencies
* Java 8
* Java 11
* Maven 3.0
* Git
* OpenCL (optional but recommended)
......@@ -56,10 +56,11 @@ To execute the `.jar` files it is highly recommended to use Oracle Java 8.
Vadere uses computer's video card to speed up some computations. Therefore, following OpenCL components should be installed:
* the OpenCL Installable Client Driver loader also called ICD loader (Opencl.dll for Windows and libOpenCL.so for Linux)
* OpenCL Drivers for your device
* drivers for your device(s)
Both should be offered by the vendor of your device (often there are also open-source solutions). The device can be a CPU as well as a GPU (recommanded). For example, if you have a NVIDIA GPU instaling your drivers should be enough to install both components.
Vadere will search for the best suiable device which is supported. On a desktop workstation this should be your video card (GPU). If there is no device supporting OpenCL Vadere will use a plain and slower Java-implementation instead.
Both should be offered by the vendor of your device. The device can be a CPU as well as a GPU (recommanded). For example if you have a NVIDIA GPU updating your drivers should be enough to install both components.
Vadere will search for the best device which is supported. On a desktop workstation this should be your video card (GPU). If there is no device Vadere will use a plain and slower Java-Implementation instead.
Please, use following instructions to set up the OpenCL components for your operating system:
* Windows: For further information using OpenCL on Windows read the paragraph Running an OpenCL application [click here](https://streamcomputing.eu/blog/2015-03-16/how-to-install-opencl-on-windows/).
......
Tools/ContinuousIntegration/collect_line_and_branch_coverage.py
View file @ 9c8b9a99
# Extract line and branch coverage (in percentage) from HTML coverage reports
# which are created by Maven's jacoco plugin.
# which are created by Maven's JaCoCo plugin.
# Use top-level pom.xml to search in correct subdirectories.
#
# Wach out: call this script from root directory of project. E.g.
......@@ -11,6 +11,10 @@ import xml.etree.ElementTree as ET
import os
import re
# Use Unix path separators because this script is usually run in a Bash-context.
# And Bash fails with Windows path separators which are introduced by os.path.join(...)
path_separator = "/"
def get_modules_from_pom_file(filename="pom.xml"):
"""Return a list of submodules which where found in passed "pom.xml"."""
......@@ -33,15 +37,23 @@ def extract_line_and_branch_coverage(module_names):
module_to_coverage = dict()
default_coverage_file = "target/site/coverage-reports/index.html"
default_coverage_file = path_separator.join(["target", "coverage-reports", "index.html"])
for module in module_names:
coverage_path = os.path.join(module, default_coverage_file)
coverage_path = path_separator.join([module, default_coverage_file])
with open(coverage_path, "r") as file:
coverage_report = file.read()
# TODO: Regex seems be be broken on Windows CI worker. Find out why!
# This is a somewhat dirty fix for windows OS (I leave the todo open, if someone knows a cleaner solution.
# When parsing the html, there appear unicode(?) issues and in this case whitespaces are represented as Â\xa0
# I think this has something to do with it:
# https://stackoverflow.com/questions/13865346/why-would-a-python-regex-compile-on-linux-but-not-windows
coverage_report = coverage_report.replace('Â\xa0', ' ')
regex_pattern = re.compile(r"Total.*?([0-9]{1,3})\s?%.*?([0-9]{1,3})\s?%")
regex_pattern = re.compile(r"Total.*?([0-9]{1,3})%.*?([0-9]{1,3})%")
match = regex_pattern.search(coverage_report)
if match:
......@@ -49,6 +61,8 @@ def extract_line_and_branch_coverage(module_names):
branch_coverage = float(match.group(2))
module_to_coverage[module] = (line_coverage, branch_coverage)
else:
print(coverage_report)
print(f"match={match}")
raise Exception("Coverage data not found for module: {}".format(module))
return module_to_coverage
......
Tools/ContinuousIntegration/run_orign_translation_test.py
View file @ 9c8b9a99
......@@ -3,8 +3,8 @@ import shutil
import subprocess
import time
from vadere_analysis_tool import VadereProject
from vadere_analysis_tool.analysis import OriginDeviation
from vadereanalysistool import VadereProject
from vadereanalysistool.analysis import OriginDeviation
def run_scenario_files_with_vadere_console(scenario_files, vadere_console="VadereSimulator/target/vadere-console.jar",
......
Tools/ContinuousIntegration/run_seed_comparison_test.d/complex_UNIT_001.scenario 0 → 100644
View file @ 9c8b9a99
{
"name" : "complex_UNIT_001",
"description" : "",
"release" : "0.6",
"processWriters" : {
"files" : [ {
"type" : "org.vadere.simulator.projects.dataprocessing.outputfile.TimestepPedestrianIdOutputFile",
"filename" : "postvis.trajectories",
"processors" : [ 1, 2 ]
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.outputfile.TimestepPedestrianIdOverlapOutputFile",
"filename" : "overlaps.csv",
"processors" : [ 3 ]
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.outputfile.NoDataKeyOutputFile",
"filename" : "overlapCount.txt",
"processors" : [ 4 ]
} ],
"processors" : [ {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.PedestrianPositionProcessor",
"id" : 1
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.PedestrianTargetIdProcessor",
"id" : 2
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.PedestrianOverlapProcessor",
"id" : 3
}, {
"type" : "org.vadere.simulator.projects.dataprocessing.processor.NumberOverlapsProcessor",
"id" : 4,
"attributesType" : "org.vadere.state.attributes.processor.AttributesNumberOverlapsProcessor",
"attributes" : {
"pedestrianOverlapProcessorId" : 3
}
} ],
"isTimestamped" : true
},
"scenario" : {
"mainModel" : "org.vadere.simulator.models.osm.OptimalStepsModel",
"attributesModel" : {
"org.vadere.state.attributes.models.AttributesOSM" : {
"stepCircleResolution" : 18,
"numberOfCircles" : 1,
"varyStepDirection" : true,
"stepLengthIntercept" : 0.4625,
"stepLengthSlopeSpeed" : 0.2345,
"stepLengthSD" : 0.036,
"movementThreshold" : 0.0,
"optimizationType" : "NELDER_MEAD",
"movementType" : "ARBITRARY",
"dynamicStepLength" : true,
"updateType" : "EVENT_DRIVEN",
"seeSmallWalls" : false,
"minimumStepLength" : false,
"targetPotentialModel" : "org.vadere.simulator.models.potential.fields.PotentialFieldTargetGrid",
"pedestrianPotentialModel" : "org.vadere.simulator.models.potential.PotentialFieldPedestrianCompactSoftshell",
"obstaclePotentialModel" : "org.vadere.simulator.models.potential.PotentialFieldObstacleCompactSoftshell",
"submodels" : [ ]
},
"org.vadere.state.attributes.models.AttributesPotentialCompactSoftshell" : {
"pedPotentialIntimateSpaceWidth" : 0.45,
"pedPotentialPersonalSpaceWidth" : 1.2,
"pedPotentialHeight" : 50.0,
"obstPotentialWidth" : 0.8,
"obstPotentialHeight" : 6.0,
"intimateSpaceFactor" : 1.2,
"personalSpacePower" : 1,
"intimateSpacePower" : 1
},
"org.vadere.state.attributes.models.AttributesFloorField" : {
"createMethod" : "HIGH_ACCURACY_FAST_MARCHING",
"potentialFieldResolution" : 0.1,
"obstacleGridPenalty" : 0.1,
"targetAttractionStrength" : 1.0,
"timeCostAttributes" : {
"standardDeviation" : 0.7,
"type" : "UNIT",
"obstacleDensityWeight" : 3.5,
"pedestrianSameTargetDensityWeight" : 3.5,
"pedestrianOtherTargetDensityWeight" : 3.5,
"pedestrianWeight" : 3.5,
"queueWidthLoading" : 1.0,
"pedestrianDynamicWeight" : 6.0,
"loadingType" : "CONSTANT"
}
}
},
"attributesSimulation" : {
"finishTime" : 90.0,
"simTimeStepLength" : 0.4,
"realTimeSimTimeRatio" : 0.0,
"writeSimulationData" : true,
"visualizationEnabled" : true,
"printFPS" : false,
"digitsPerCoordinate" : 2,
"useFixedSeed" : true,
"fixedSeed" : -3213925745664992646,
"simulationSeed" : 0
},
"topography" : {
"attributes" : {
"bounds" : {
"x" : 0.0,
"y" : 0.0,
"width" : 50.0,
"height" : 50.0
},
"boundingBoxWidth" : 0.5,
"bounded" : true
},
"obstacles" : [ {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 0.4,
"y" : 0.4
}, {
"x" : 0.5,
"y" : 18.7
}, {
"x" : 9.8,
"y" : 18.8
}, {
"x" : 15.4,
"y" : 8.3
}, {
"x" : 15.4,
"y" : 0.5
} ]
},
"id" : 9
}, {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 0.6,
"y" : 34.9
}, {
"x" : 0.4,
"y" : 49.5
}, {
"x" : 14.9,
"y" : 49.5
}, {
"x" : 14.7,
"y" : 42.3
}, {
"x" : 9.0,
"y" : 35.0
} ]
},
"id" : 10
}, {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 36.0,
"y" : 25.7
}, {
"x" : 12.4,
"y" : 25.8
}, {
"x" : 12.5,
"y" : 29.5
}, {
"x" : 20.7,
"y" : 39.6
}, {
"x" : 32.0,
"y" : 39.5
}, {
"x" : 36.1,
"y" : 32.7
} ]
},
"id" : 11
}, {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 19.8,
"y" : 22.1
}, {
"x" : 32.3,
"y" : 22.2
}, {
"x" : 33.24321348467342,
"y" : 18.125317746210836
}, {
"x" : 43.4,
"y" : 18.2
}, {
"x" : 43.5,
"y" : 3.9
}, {
"x" : 35.9,
"y" : 3.9
}, {
"x" : 29.8,
"y" : 11.3
}, {
"x" : 29.800000000000008,
"y" : 11.311504424778763
}, {
"x" : 19.8,
"y" : 11.4
} ]
},
"id" : 12
}, {
"shape" : {
"x" : 35.5,
"y" : 39.3,
"width" : 14.3,
"height" : 10.2,
"type" : "RECTANGLE"
},
"id" : 5
} ],
"stairs" : [ {
"shape" : {
"type" : "POLYGON",
"points" : [ {
"x" : 13.0,
"y" : 29.8
}, {
"x" : 7.9,
"y" : 33.7
}, {
"x" : 14.8,
"y" : 42.1
}, {
"x" : 20.3,
"y" : 38.8
} ]
},
"id" : 20,
"treadCount" : 35,
"upwardDirection" : {
"x" : 1.0,
"y" : 1.3
}
}, {
"shape" : {
"x" : 43.1,
"y" : 3.9,
"width" : 6.4,
"height" : 14.4,
"type" : "RECTANGLE"
},
"id" : 14,
"treadCount" : 45,
"upwardDirection" : {
"x" : 0.0,
"y" : 1.0
}
} ],
"targets" : [ {
"id" : 6,
"absorbing" : false,
"shape" : {
"x" : 41.6,
"y" : 20.0,
"width" : 3.7,
"height" : 10.0,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
}, {
"id" : 7,
"absorbing" : true,
"shape" : {
"x" : 19.9,
"y" : 41.7,
"width" : 10.0,
"height" : 3.7,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
}, {
"id" : 8,
"absorbing" : true,
"shape" : {
"x" : 20.0,
"y" : 4.6,
"width" : 10.0,
"height" : 3.0,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
}, {
"id" : 15,
"absorbing" : false,
"shape" : {
"x" : 5.2,
"y" : 20.0,
"width" : 3.5,
"height" : 9.8,
"type" : "RECTANGLE"
},
"waitingTime" : 0.0,
"waitingTimeYellowPhase" : 0.0,
"parallelWaiters" : 0,
"individualWaiting" : true,
"deletionDistance" : 0.1,
"startingWithRedLight" : false,
"nextSpeed" : -1.0
} ],
"sources" : [ {
"id" : 2,
"shape" : {
"x" : 45.8,
"y" : 20.0,
"width" : 3.7,
"height" : 10.0,
"type" : "RECTANGLE"
},
"interSpawnTimeDistribution" : "org.vadere.state.scenario.ConstantDistribution",
"distributionParameters" : [ 1.0 ],
"spawnNumber" : 3,
"maxSpawnNumberTotal" : -1,
"startTime" : 0.0,
"endTime" : 15.0,
"spawnAtRandomPositions" : true,
"useFreeSpaceOnly" : true,
"targetIds" : [ 8 ],
"groupSizeDistribution" : [ 1.0 ],
"dynamicElementType" : "PEDESTRIAN"
}, {
"id" : 3,
"shape" : {
"x" : 20.0,
"y" : 45.8,
"width" : 10.0,
"height" : 3.7,
"type" : "RECTANGLE"
},
"interSpawnTimeDistribution" : "org.vadere.state.scenario.ConstantDistribution",
"distributionParameters" : [ 1.0 ],
"spawnNumber" : 3,
"maxSpawnNumberTotal" : -1,
"startTime" : 0.0,
"endTime" : 15.0,
"spawnAtRandomPositions" : true,
"useFreeSpaceOnly" : true,
"targetIds" : [ 15, 8 ],
"groupSizeDistribution" : [ 1.0 ],
"dynamicElementType" : "PEDESTRIAN"
}, {
"id" : 4,
"shape" : {
"x" : 20.0,
"y" : 0.4,
"width" : 10.0,
"height" : 3.7,
"type" : "RECTANGLE"
},
"interSpawnTimeDistribution" : "org.vadere.state.scenario.ConstantDistribution",
"distributionParameters" : [ 1.0 ],
"spawnNumber" : 3,
"maxSpawnNumberTotal" : -1,
"startTime" : 0.0,