Commit a43c47e7 authored by Eckhart Arnold's avatar Eckhart Arnold
Browse files

ts2dataclass example extended

parent 5d8499a3
......@@ -457,6 +457,8 @@ terms of speed and considerably simplified abtract-syntax-tree generation
stage, because most of the unnecessary structure of concrete-syntax-trees
has already been eliminated at the parsing stage.
.. _comments_and_whitespace:
Comments and Whitespace
-----------------------
......
......@@ -262,8 +262,8 @@ yields::
phrase "Buckingham Palace"
Using XML-Tools with DHParser
-----------------------------
Connecting DHParser with XML-technology
---------------------------------------
Although DHParser offers rich support for tree-transformation, he
whish may arise to use standard XML-tools for tree-transformation
......
......@@ -219,10 +219,10 @@ Larger scale DSL-projects
Larger and more complex DSL-projects can easily be setup by calling the "dhparser"-script
with a name of a project-directory that will then be created and filled with some templates::
$ dhparser json
$ cd json
$ dhparser JSON
$ cd JSON
$ dir
example.dsl json.ebnf jsonServer.py README.md tests_grammar tst_json_grammar.py
example.dsl JSON.ebnf JSONServer.py README.md tests_grammar tst_JSON_grammar.py
The first step is to replace the ".ebnf"-file that contains a simple demo-grammar with your
own grammar. For the sake of the example we'll write our json-Grammar into this file::
......@@ -278,7 +278,7 @@ has changed and runs the unit tests in the ``tests_grammar`` subdirectory.
After filling in the above grammar in the ``json.ebnf``-file, a parser can
be generated by running the test skript::
$ python tst_json_grammar.py
$ python tst_JSON_grammar.py
If there were no errors, a new ``jsonParser.py`` appears in the directory.
Before we can try it, we need some test-data. Then we can run the script
......@@ -286,7 +286,7 @@ just like before::
$ rm example.dsl
$ echo '{ "one": 1, "two": 2 }' >example.json
$ jsonParser.py --xml example.json
$ python JSONParser.py --xml example.json
<json>
<object>
...
......@@ -303,17 +303,19 @@ To reach this goal DHParser follows a few, mostly intuitive, conventions:
side will carry the symbol as a tag-name.
2. Some symbols can explicitly be marked as "disposable" (or "insignificant"),
however. These may not appear in the syntax-tree and nodes of their type
type may be replaced by their children, silently. Thus, you'll never see
an "_elment"-node in a JSON-syntaxtree produced by the above grammar,
but only object-, array-, string-, number-, true-, false- or null-nodes.
however. A node generated by the parser associated with a disposable symbol
will not appear in the syntax-tree but be replaced by its children, silently.
Thus, you'll never see an "_elment"-node in a JSON-syntaxtree produced
by the above grammar, but only object-, array-, string-, number-, true-,
false- or null-nodes. (See :py:ref:`~ebnf.simplifying_syntax_trees`.)
3. Insignificant whitespace is denoted by the single character ``~``.
3. Insignificant whitespace is denoted a the single character: ``~``.
4. Comments defined by the ``@comment``-directive at the top of the grammar
are allowed in any place where insignificant ``~``-whitespace is
allowed. Thus, you never need to worry about where to provide for
comments in you grammar. It is as easy as it is intuitive.
(See :py:ref:`~ebnf.comments_and_whitespace`.)
5. Delimiters like "," or "[", "]" etc. typically appear as literal
string values in a grammar. Now, since delimiters are typically
......@@ -338,29 +340,99 @@ To reach this goal DHParser follows a few, mostly intuitive, conventions:
non-disposable symbol, e.g. ``opening_bracket = "("`` and using
this symbol instead of the string literal in other expressions.
7. Ah, and yes, of course you do not need to end grammar definitions
7. Ah, and yes, of course, you do not need to end grammar definitions
with a semicolon ``;`` as demanded by the ISO-norm for EBNF :-)
Declarative Abstract-syntax-tree transformation
-----------------------------------------------
DHParser does does not hide any stages of the tree generation
process. Thus, you get full access to the (simplified) concrete
syntax tree (CST) as well as to the abstract syntax tree (AST).
Abstract syntax tree generation is controlled in
declarative style by simple lists of transformations
applied to each node depending on its type. Remember
our JSON-example from above? In the simplified
concrete syntax tree string-objects still contained the
quotation mark delimiting the string. Since these are not
needed in the data you'd like to retrieve from a JSON-file,
let's remove them from the abstract syntax-tree::
JSON_AST_transformation_table = {
"string": [remove_brackets]
}
The ``JSON_AST_transformation_table``-dictionary can
be found in the generated ``JSONParser.py``-script.
Simply add the rule "remove_bracket" from the
:py:mod:`transform`-module to the list of rules
for those nodes where you wish to remove any delimiters
at the beginning or end::
$ python JSONParser.py --xml example.json
<json>
<object>
<member>
<string>
<PLAIN>one</PLAIN>
</string>
...
Alternatively, you could also have used the rule
``"string": [remove_children(':Text')]`` in case you
are sure that nodes with the tag-name ":Text" can
only occur in a string at the beginning and at the
end as nodes containing the quotation mark-delimiters
of that string.
The :py:mod:`transform`-module
contains a number of useful transformation-rules
that can be combined almost arbitrarily in order
to reshape the concrete syntax-tree and carve
out the abstract syntax tree. However, if the
grammar is well-designed and if the
concrete syntax tree has already been simplified
with the help of DHParser's ``@disposable``-,
``@reduction``- and ``@drop``-directives, only
transformations should be necessary to produce
the abstract syntax-tree.
In specific application cases it is often desirable
to model the abstract syntax-tree as a tree of
objects of different classes. However, since DHParser
is a generic Parser-generator, DHParser's syntax-trees
are composed of a single :py:class:`~syntaxtree.Node`-type.
Nodes contain either text-data or have one or more other nodes
as children (but not both). The "kind" or "type"
of a node is indicated by its "tag-name". It should be
easy, though, to this into an application-specific
tree of objects of different classes.
Test-driven grammar development
-------------------------------
Invoking the post-mortem debugger
---------------------------------
Tree compilation with the Visitor-pattern
-----------------------------------------
Tracing post-mortem debugger for parsers
----------------------------------------
Fail-tolerant parsing
---------------------
Running the DSL-parser as a server
----------------------------------
Tree compilation with the Visitor-pattern
-----------------------------------------
Serialization as you like it: JSON, XML, S-expressions
------------------------------------------------------
Connecting with XML-technolgies easily
--------------------------------------
Server-mode for setting up DSL-language-servers
-----------------------------------------------
Performance considerations
--------------------------
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment