Getting rid of the old package_dir
Summary
The old package_dir
was problematic for a consistend (de)serialization of Timor objects. This enhancement introduces a new enforced file and JSON structure to implicitly infer asset locations.
Roadmap
- A fixed and programmatically enforced structure for modulesDB json-file and asset-file locations:
parent_folder_defining_db_name
|
|–– modules.json
|-- assets (arbitrary name)
(additional files can be contained in this folder)
- Within the modules.json, the "file" keyowrd for assets defines the asset path relative to
parent_folder_defining_db_name
- When loading a DB from file, the absolut filepath will be inferred from the
modules.json
file path. During runtime, Timor will only work with the aboslute file path - When writing a DB to file, the relative filepath will be generated automatically by "subtracting" the filepath for the modules.json
- If the assets referred to in modules.json do not exist at the newly inferred filepath, the config will define how to handle this case by default. You can also provide this as a method argument to override the default. Possible options will be ("ignore", "copy", "fail", and "symlink"). This will be defined in the [SERIALIZING] section under keyword "db_assets_copy_behavior"
- Similarly, we define the following enforced file structure for tasks:
parent_folder (arbitrary name)
|
|-- assets (arbitrary name)
|-- task_x.json
|-- task_y.json
|-- namespace_folder
|-- task_a.json
|-- task_b.json
(additional files can be placed in any of these folders)
- The filename of the task MUST match its ID
- If a task is placed in a namespace folder, it's ID will look like
namespace_folder/own_id
- Assets for this task are defined relative to the
parent_folder
, the absolute filepath will be inferred from task location similarly to the procedure for DBs - The config parameter in [SERIALIZING] section under keyword "task_assets_copy_behavior" defines how to deal with the case of non-existing assets when saving a runtime task to a new location
Implications
- You can't any longer to DB.from_json_data(json.loads(...)) because you need to infer the absolute filepath from the file location
- The "file" keyword in any jsonable will be treated different from the others because we expect to have to resolve the absolute filepath there
- slashes "/" can not be used for "normal" naming of task IDs
- ID's of tasks are not allowed to start with "/"
- Absolute paths in serialized jsons ("/a/b/c") will raise an exception
Affected JSONables
- ModulesDB
- AtomicModule
- Task
/cc @JonathanKuelz @ga59peh