Commit 77e303d9 authored by Stefan Schuhbaeck's avatar Stefan Schuhbaeck
Browse files

Merge branch 'master' into assertionLog

parents e2184ebf 40ef5847
Pipeline #88002 canceled with stages
......@@ -41,7 +41,6 @@ unit_tests_with_coverage:
stage: test
script:
- Documentation/version-control/git-hook-vadere-software
- mvn clean
- mvn -Dtest=!TestConvolution,!TestBitonicSort,!TestCLLinkedList,!TestCLOptimalStepsModel test
- python3 Tools/ContinuousIntegration/collect_line_and_branch_coverage.py
......@@ -54,7 +53,6 @@ run_scenario_files:
stage: deploy
script:
- Documentation/version-control/git-hook-vadere-software
- mvn -Dmaven.test.skip=true package
- python3 Tools/ContinuousIntegration/run_vadere_console_with_all_scenario_files.py
artifacts:
......@@ -78,7 +76,6 @@ run_seed_test:
stage: deploy
script:
- Documentation/version-control/git-hook-vadere-software
- mvn clean
- mvn -Dmaven.test.skip=true package
- python3 Tools/VadereAnalysisTools/VadereAnalysisTool/setup.py install --user
......
......@@ -27,6 +27,26 @@
sub-command. This will will turn the ScenarioChecker on or off for the command
line. If the Checker detects an error the simulation will not be executed.
- `TopographyCreator` added functionalities:
- move all topography elements by some dx, dy [issue#171](https://gitlab.lrz.de/vadere/vadere/issues/171)
- move the whole topography by some dx, dy [issue#148](https://gitlab.lrz.de/vadere/vadere/issues/148)
- automatically merge overlapping obstacles by using the weiler algorithm.
- `Processors`:
- new processors to compute the fundamental diagrams (by using different velocity, flow and density measurement methods)
- `GUI`:
- improved coloring
- `OutputFile`: Distinguish between indices (rows) and headers (columns), in code and with a new checkbox that when enables allow to write meta-data into output files. Furthermore, resolve naming conflicts (if they occur) in headers of output files (see #201).
### Changed
- Removed directory `Documentation/version-control` which contained the Git hooks. The Git hooks are not required anymore. Instead, added `git rev-parse HEAD` to file `VadereSimulator/pom.xml` to create `VadereSimulator/resources/current_commit_hash.txt` during each build via `mvn compile`.
**Note:** The file `current_commit_hash.txt` is created during Maven's validation phase, i.e., before the actual build.
`
## v0.6 (2018-09-07)
### Added
......
Copy files "post-checkout", "post-merge", "post-applypatch" and "post-commit" to .git/hooks/
The file "current_commit_hash.txt" will be created after a change to the current commit status. To create this file without changes to the current commit, you can choose "Switch/Checkout..." on the Repository folder.
\ No newline at end of file
#!/bin/sh
exec git log -1 --format=format:"%H" > VadereSimulator/resources/current_commit_hash.txt
\ No newline at end of file
#!/bin/sh
exec Documentation/version-control/git-hook-vadere-software
\ No newline at end of file
#!/bin/sh
exec Documentation/version-control/git-hook-vadere-software
\ No newline at end of file
#!/bin/sh
exec Documentation/version-control/git-hook-vadere-software
\ No newline at end of file
#!/bin/sh
exec Documentation/version-control/git-hook-vadere-software
\ No newline at end of file
......@@ -31,6 +31,8 @@ The Vadere framework includes a mesh generator for unstructured high-quality 2D
* Git
* OpenCL
**Note:** Please, ensure that the Git executable can be found in the `PATH` variable of your operating system.
### Install OpenCL
Vadere uses computer's video card to speed up some computations. Therefore, following OpenCL components must be installed:
......@@ -62,27 +64,14 @@ With the following steps, you can run a simulation with one of the built-in exam
- choose `vadere.project` of one of the projects e.g. [TestOSM](VadereModelTests/TestOSM) and click *open*
- select the scenario on the left and press *run selected scenario*
### Use Git Hooks
Since it is important to reproduce simulation results, we have the guidline that each output file has to provide its commit-hash. This commit-hash identifies
the state the software was in when the output file was generated. Therefore, git hooks save the commit-hash in the **current_commit_hash.txt** which
will be created in the [VadereSimulator/resource](vadere/VadereSimulator/resource) directory whenever a commit is made or the developer
switches to another commit. If the file is missing or there is no commit-hash in it, you will be warned in the log.
We strongly suggest that you install these git hooks on your local machine:
1. Copy files [post-checkout](Documentation/version-control/post-checkout), [post-merge](Documentation/version-control/post-merge),
[post-applypatch](Documentation/version-control/post-applypatch) and [post-commit](Documentation/version-control/post-commit)
from [version-control](Documentation/version-control) to your local **.git/hooks/** directory.
These files start the script [git-hook-vadere-software](Documentation/version-control/git-hook-vadere-software).
2. Make sure that [git-hook-vadere-software](Documentation/version-control/git-hook-vadere-software) is executable.
To create the **current_commit_hash.txt** without changes to the current commit, you can choose *Switch/Checkout...* on the Repository folder or you
switch to another branch and switch back again using the command line or any other tool you prefer.
## Changelog
See [CHANGELOG.md](CHANGELOG.md) for a list of changes.
## JavaDoc
- [state](http://www.vadere.org/javadoc/state/index.html)
## Contribution
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to set up the development environment and the coding guidelines.
......
# In the given Vadere topography file, filter out all obstacles which are
# outside a boundary box (given by lower left and upper right corner).
#
# A Vadere topography looks like this:
#
# {
# ...,
# "obstacles" : [
# {
# "shape" : {
# "type" : "POLYGON",
# "points" : [
# { "x" : 295.1790000000037, "y" : 193.12900000065565 },
# { "x" : 299.71000000002095, "y" : 195.3730000006035 },
# { "x" : 300.445000000007, "y" : 193.89099999982864 },
# { "x" : 295.91399999998976, "y" : 191.6469999998808 },
# { "x" : 295.1790000000037, "y" : 193.12900000065565 } ]
# },
# "id" : -1
# } ],
# "stairs" : [ ],
# "targets" : [ ],
# "sources" : [ ],
# ...
# }
from string import Template
import argparse
import json
import math
def parse_command_line_arguments():
parser = argparse.ArgumentParser(description="Filter out all obstacles which are outside a boundary box (given by lower left and upper right corner).")
parser.add_argument("filename", type=str,
help="A Vadere topography file.",
)
parser.add_argument("boundary_box", type=json.loads,
help="Boundary box consisting of lower left and upper right corner (syntax: '{\"x1\": 0.0, \"y1\": 0.0, \"x2\": 10.0, \"y2\": 10.0}')",
)
parser.add_argument("-o", "--output", type=str, nargs="?",
help="Specify filename to write the output to file instead of stdout.")
args = parser.parse_args()
raise_exception_on_invalid_boundary_box(args.boundary_box)
return args
def raise_exception_on_invalid_boundary_box(boundary_box):
expected_keys = set(["x1", "y1", "x2", "y2"])
actual_keys = boundary_box.keys()
is_no_empty_set = bool(expected_keys.symmetric_difference(actual_keys))
if is_no_empty_set:
raise ValueError("Given boundary box does not contain required keys: {}".format(expected_keys))
if boundary_box["x2"] <= boundary_box["x1"]:
raise ValueError("Invalid boundary box: x2 <= x1 ({} <= {})".format(boundary_box["x2"], boundary_box["x1"]))
if boundary_box["y2"] <= boundary_box["y1"]:
raise ValueError("Invalid boundary box: y2 <= y1 ({} <= {})".format(boundary_box["y2"], boundary_box["y1"]))
def parse_vadere_topography_file(filename):
with open(filename, "r") as json_file:
return json.load(json_file)
def filter_out_obstacles_outside_boundary_box(vadere_topography, boundary_box):
# Note: Obstacles crossing the boundary box are NOT filtered out.
obstacles = vadere_topography["obstacles"]
polygon_obstacles = [obstacle for obstacle in obstacles if obstacle["shape"]["type"].lower() == "polygon"]
obstacles_inside_boundary_box = []
for polygon_obstacle in polygon_obstacles:
vertices = polygon_obstacle["shape"]["points"]
obstacle_is_inside_boundary = False
for vertex in vertices:
inside_x_boundary = vertex["x"] >= boundary_box["x1"] and vertex["x"] <= boundary_box["x2"]
inside_y_boundary = vertex["y"] >= boundary_box["y1"] and vertex["y"] <= boundary_box["y2"]
if inside_x_boundary and inside_y_boundary:
obstacle_is_inside_boundary = True
break
if obstacle_is_inside_boundary:
obstacles_inside_boundary_box.append(polygon_obstacle)
return obstacles_inside_boundary_box
def convert_obstacles_to_list(obstacles):
# Convert "obstacles" into a list of lists. I.e., result[0] returns a list
# of vertices for first obstacle (a vertice is another list containing two
# elements). This method is necessary to use methods the following methods
# (e.g., find_minimal_coordinates(obstacles) which expect a list of lists.
obstacles_as_list = []
for obstacle in obstacles:
vertices = obstacle["shape"]["points"]
# Create a list of two elements for each vertice. All vertices are
# combined into one single list by list comprehension.
vertices_as_list = [[vertice["x"], vertice["y"]] for vertice in vertices]
obstacles_as_list.append(vertices_as_list)
return obstacles_as_list
def find_minimal_coordinates(obstacles):
all_vertices = [vertex for obstacle in obstacles for vertex in obstacle]
tuple_with_min_x = min(all_vertices, key=lambda point: point[0])
tuple_with_min_y = min(all_vertices, key=lambda point: point[1])
return (tuple_with_min_x[0], tuple_with_min_y[1])
def find_maximum_coordinates(obstacles):
all_vertices = [vertex for obstacle in obstacles for vertex in obstacle]
tuple_with_max_x = max(all_vertices, key=lambda point: point[0])
tuple_with_max_y = max(all_vertices, key=lambda point: point[1])
rounded_max_x = math.ceil(tuple_with_max_x[0])
rounded_max_y = math.ceil(tuple_with_max_y[1])
return (rounded_max_x, rounded_max_y)
def shift_obstacles(obstacles, shift_in_x_direction, shift_in_y_direction):
shifted_obstacles = []
for obstacle in obstacles:
shifted_obstacle = \
[(vertex[0] + shift_in_x_direction, vertex[1] + shift_in_y_direction) for vertex in obstacle]
shifted_obstacles.append(shifted_obstacle)
return shifted_obstacles
def convert_obstacles_to_vadere_obstacle_strings(obstacles):
list_of_vadere_obstacles_as_strings = []
# A single obstacle is a list of vertices.
for obstacle in obstacles:
vadere_obstacle_as_string = create_vadere_obstacle_from_vertices(obstacle)
list_of_vadere_obstacles_as_strings.append(vadere_obstacle_as_string)
return list_of_vadere_obstacles_as_strings
def create_vadere_obstacle_from_vertices(vertices):
vadere_obstacle_string = """{
"shape" : {
"type" : "POLYGON",
"points" : [ $points ]
},
"id" : -1
}"""
vadere_point_string = '{ "x" : $x, "y" : $y }'
obstacle_string_template = Template(vadere_obstacle_string)
point_string_template = Template(vadere_point_string)
points_as_string = [point_string_template.substitute(x=x, y=y) for x, y in vertices]
points_as_string_concatenated = ",\n".join(points_as_string)
vadere_obstacle_as_string = obstacle_string_template.substitute(points=points_as_string_concatenated)
return vadere_obstacle_as_string
def create_vadere_topography_with_obstacles(obstacles, width, height):
with open("vadere_topography_template.txt", "r") as template_file:
vadere_topography_template = template_file.read()
vadere_topography_string = Template(vadere_topography_template).substitute(obstacles=obstacles, width=width, height=height)
return vadere_topography_string
def write_parsing_statistics(filename, boundary_box, all_obstacles, obstacles_inside_boundary_box, shift_obstacles_by):
print("Filename: {}".format(filename))
print(" Boundary box: {}".format(boundary_box))
print(" All obstacles: {}".format(len(all_obstacles)))
print(" Obstacles inside boundary box: {}".format(len(obstacles_inside_boundary_box)))
print(" Shift obstacles by (x, y) for simulation: {}".format(shift_obstacles_by))
def write_vadere_topography_string_to(vadere_topography_string, output_filename):
if output_filename == None:
print(vadere_topography_string)
else:
with open(output_filename, "w") as text_file:
print(vadere_topography_string, file=text_file)
if __name__ == "__main__":
args = parse_command_line_arguments()
vadere_topography = parse_vadere_topography_file(args.filename)
obstacles_inside_boundary_box = filter_out_obstacles_outside_boundary_box(vadere_topography, args.boundary_box)
obstacles_inside_boundary_box_as_list = convert_obstacles_to_list(obstacles_inside_boundary_box)
shift_obstacles_by = find_minimal_coordinates(obstacles_inside_boundary_box_as_list)
shifted_obstacles = shift_obstacles(obstacles_inside_boundary_box_as_list, shift_in_x_direction=-shift_obstacles_by[0], shift_in_y_direction=-shift_obstacles_by[1])
topography_width, topography_height = find_maximum_coordinates(shifted_obstacles)
vadere_obstacles_as_strings = convert_obstacles_to_vadere_obstacle_strings(shifted_obstacles)
vadere_obstacles_as_strings_concatenated = ",\n".join(vadere_obstacles_as_strings)
vadere_topography_string = create_vadere_topography_with_obstacles(vadere_obstacles_as_strings_concatenated, topography_width, topography_height)
write_parsing_statistics(args.filename, args.boundary_box, vadere_topography["obstacles"], obstacles_inside_boundary_box, shift_obstacles_by)
write_vadere_topography_string_to(vadere_topography_string, args.output)
# Convert a GeoJSON file to a Vadere topology (in Cartesian coordinates).
#
# Steps to run this script:
#
# 1. Go to https://opmops.virtualcitysystems.de/dss/kaiserslautern/#/
# 2. Click "Weitere Funktionen" icon -> "Export" icon -> "Vektor" -> "ALK Gebäudegrundrisse" -> "Weiter".
# 3. Adjust region of intereset and click "Weiter".
# 4. Set "Ausgabformat" = JSON and "Rückgabe" = EPSG:25832, and click "Anfrage senden".
# 3. Call script and pass exported file from (4):
#
# python3 <script> <exported_file>
#
# 4. Insert output into "topography" tab of Vadere.
#
# Watch out: before running this script, install its dependencies using pip:
#
# pip install -r requirements.txt
#
# Example GeoJSON file:
#
# {
# "type":"FeatureCollection",
# "totalFeatures":332,
# "features":[
# {
# "type":"Feature",
# "id":"Gebaeude.45",
# "geometry":{
# "type":"MultiPolygon",
# "coordinates":[
# [
# [
# [
# 410260.098,
# 5476786.396
# ],
# ...
# ]
# ]
# ]
# },
# "geometry_name":"the_geom",
# "properties":{
# "LAYER":"Unknown Area Type",
# "ELEVATION":0,
# "ID":0
# }
# },
# ...
# ],
# "crs":{
# "type":"name",
# "properties":{
# "name":"urn:ogc:def:crs:EPSG::25832"
# }
# }
# }
#
#
# A Vadere obstacle looks like this:
#
# {
# "shape" : {
# "type" : "POLYGON",
# "points" : [ { "x" : 43.7, "y" : 3.4 }, ... ]
# },
# "id" : -1
# }
from string import Template
import argparse
import json
import math
def parse_command_line_arguments():
parser = argparse.ArgumentParser(description="Convert a GeoJSON file to a Vadere topology description.")
parser.add_argument("filename", type=str, nargs="?",
help="A GeoJSON file.",
default="maps/map_kaiserslautern_pollichstraße.json",
)
parser.add_argument("-o", "--output", type=str, nargs="?",
help="Specify filename to write the output to file instead of stdout.")
args = parser.parse_args()
return args
def parse_geojson_file(filename):
with open(filename, "r") as json_file:
return json.load(json_file)
def filter_for_buildings(geojson_content):
# Top-level keys of a GeoJSON file: type, totalFeatures, features, crs
buildings = []
for feature in geojson_content["features"]:
geometry = feature["geometry"]
if geometry["type"] == "MultiPolygon":
# Watch out: the coordinates of a building are wrapped within two lists (each has length one).
vertices_of_building_as_a_single_list = geometry["coordinates"][0][0]
buildings.append(vertices_of_building_as_a_single_list)
return buildings
def find_minimal_coordinates(buildings):
all_vertices = [vertex for building in buildings for vertex in building]
tuple_with_min_x = min(all_vertices, key=lambda point: point[0])
tuple_with_min_y = min(all_vertices, key=lambda point: point[1])
return (tuple_with_min_x[0], tuple_with_min_y[1])
def find_maximum_coordinates(buildings):
all_vertices = [vertex for building in buildings for vertex in building]
tuple_with_max_x = max(all_vertices, key=lambda point: point[0])
tuple_with_max_y = max(all_vertices, key=lambda point: point[1])
rounded_max_x = math.ceil(tuple_with_max_x[0])
rounded_max_y = math.ceil(tuple_with_max_y[1])
return (rounded_max_x, rounded_max_y)
def shift_buildings(buildings, shift_in_x_direction, shift_in_y_direction):
shifted_buildings = []
for building in buildings:
shifted_building = \
[(vertex[0] + shift_in_x_direction, vertex[1] + shift_in_y_direction) for vertex in building]
shifted_buildings.append(shifted_building)
return shifted_buildings
def convert_buildings_to_vadere_obstacle_strings(buildings):
list_of_vadere_obstacles_as_strings = []
# A single building is a list of vertices.
for building in buildings:
vadere_obstacle_as_string = create_vadere_obstacle_from_vertices(building)
list_of_vadere_obstacles_as_strings.append(vadere_obstacle_as_string)
return list_of_vadere_obstacles_as_strings
def create_vadere_obstacle_from_vertices(vertices):
vadere_obstacle_string = """{
"shape" : {
"type" : "POLYGON",
"points" : [ $points ]
},
"id" : -1
}"""
vadere_point_string = '{ "x" : $x, "y" : $y }'
obstacle_string_template = Template(vadere_obstacle_string)
point_string_template = Template(vadere_point_string)
points_as_string = [point_string_template.substitute(x=x, y=y) for x, y in vertices]
points_as_string_concatenated = ",\n".join(points_as_string)
vadere_obstacle_as_string = obstacle_string_template.substitute(points=points_as_string_concatenated)
return vadere_obstacle_as_string
def create_vadere_topography_with_obstacles(obstacles, width, height):
with open("vadere_topography_template.txt", "r") as template_file:
vadere_topography_template = template_file.read()
vadere_topography_string = Template(vadere_topography_template).substitute(obstacles=obstacles, width=width, height=height)
return vadere_topography_string
def write_parsing_statistics(filename, coordinate_system_as_epsg_code, boundary_box, buildings, shift_buildings_by):
print("Filename: {}".format(filename))
print(" Coordinate system: {}".format(coordinate_system_as_epsg_code))
print(" Boundary box: {}".format(boundary_box))
print(" Found buildings: {}".format(len(buildings)))
print(" Shift buildings by (x, y) for simulation: {}".format(shift_buildings_by))
def write_vadere_topography_string_to(vadere_topography_string, output_filename):
if output_filename == None:
print(vadere_topography_string)
else:
with open(output_filename, "w") as text_file:
print(vadere_topography_string, file=text_file)
if __name__ == "__main__":
args = parse_command_line_arguments()
geojson_content = parse_geojson_file(args.filename)
# A single building is a list of vertices.
buildings = filter_for_buildings(geojson_content)
# Find minimal coordinates to shift to (0,0) to be able to run simulation in Vadere.
minimal_coordinates = find_minimal_coordinates(buildings)
shifted_buildings = shift_buildings(buildings, shift_in_x_direction=-minimal_coordinates[0], shift_in_y_direction=-minimal_coordinates[1])
topography_width, topography_height = find_maximum_coordinates(shifted_buildings)
vadere_obstacles_as_strings = convert_buildings_to_vadere_obstacle_strings(shifted_buildings)
vadere_obstacles_as_strings_concatenated = ",\n".join(vadere_obstacles_as_strings)
vadere_topography_string = create_vadere_topography_with_obstacles(vadere_obstacles_as_strings_concatenated, topography_width, topography_height)
write_parsing_statistics(args.filename, geojson_content["crs"], geojson_content["bbox"], buildings, minimal_coordinates)
write_vadere_topography_string_to(vadere_topography_string, args.output)
import unittest
import filter_vadere_obstacles as under_test
class TestFilterVadereObstacles(unittest.TestCase):
def test_raise_exception_on_invalid_boundary_box_raises_value_error_on_empty_boundary_box(self):
with self.assertRaises(ValueError):
boundary_box = {}
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_value_error_if_boundary_box_contains_wrong_keys(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2": 1.0, "z": 1.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_value_error_if_boundary_box_contains_too_less_keys(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2": 1.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_value_error_if_boundary_box_contains_too_much_keys(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2": 1.0 , "y2": 1.0, "z": 0.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_error_if_x2_less_than_x1(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2": -1.0 , "y2": 1.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_error_if_x2_equals_x1(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2": 0.0 , "y2": 1.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_error_if_y2_less_than_y1(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2":1.0 , "y2": -1.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)
def test_raise_exception_on_invalid_boundary_box_raises_error_if_y2_equals_y1(self):
with self.assertRaises(ValueError):
boundary_box = { "x1": 0.0, "y1": 0.0, "x2":1.0 , "y2": 0.0 }
under_test.raise_exception_on_invalid_boundary_box(boundary_box)