10.12., 9:00 - 11:00: Due to updates GitLab may be unavailable for some minutes between 09:00 and 11:00.

Commit ffc6a53c authored by eckhart's avatar eckhart

small optimizations

parent 0bf903b8
......@@ -25,8 +25,8 @@ bigger projects, below:
Ideas for further development
=============================
Better error reporting
----------------------
Better error reporting I
------------------------
A problem with error reporting consists in the fact that at best only the very
first parsing error is reported accurately and then triggers a number of pure
......@@ -49,6 +49,15 @@ left recursion stack, etc. without making the parser guard (see
Also, a good variety of test cases would be desirable.
Better error reporting II
-------------------------
Yet another means to improve error reporting would be to supplement
the required operator "&" with an its forbidden operator, say "!&"
that would raise an error message, if some parser matches at a place
where it really shouldn't. [Add some examples here.]
Optimizations
-------------
......
......@@ -39,6 +39,7 @@ __all__ = ('ParserBase',
'MockParser',
'ZombieParser',
'ZOMBIE_PARSER',
'ZOMBIE_NODE',
'Node',
'mock_syntax_tree',
'flatten_sxpr')
......@@ -724,6 +725,9 @@ class Node(collections.abc.Sized):
return sum(child.tree_size() for child in self.children) + 1
ZOMBIE_NODE = Node(ZOMBIE_PARSER, '')
def mock_syntax_tree(sxpr):
"""
Generates a tree of nodes from an S-expression. The main purpose of this is
......
......@@ -30,7 +30,7 @@ for CST -> AST transformations.
import inspect
from functools import partial, reduce, singledispatch
from DHParser.syntaxtree import Node, WHITESPACE_PTYPE, TOKEN_PTYPE, MockParser
from DHParser.syntaxtree import Node, WHITESPACE_PTYPE, TOKEN_PTYPE, MockParser, ZOMBIE_NODE
from DHParser.toolkit import expand_table, smart_list, re, typing
from typing import AbstractSet, Any, ByteString, Callable, cast, Container, Dict, \
List, Sequence, Union, Text
......@@ -264,11 +264,12 @@ def traverse(root_node: Node,
nonlocal cache
node = context[-1]
if node.children:
context.append(ZOMBIE_NODE)
for child in node.result:
context.append(child)
context[-1] = child
traverse_recursive(context) # depth first
node.error_flag = max(node.error_flag, child.error_flag) # propagate error flag
context.pop()
context.pop()
key = key_func(node)
try:
......@@ -453,26 +454,26 @@ def _reduce_child(node: Node, child: Node):
node.result = child.result
def _pick_child(context: List[Node], criteria: CriteriaType):
"""Returns the first child that meets the criteria."""
if isinstance(criteria, int):
try:
return context[-1].children[criteria]
except IndexError:
return None
elif isinstance(criteria, str):
for child in context[-1].children:
if child.tag_name == criteria:
return child
return None
else: # assume criteria has type ConditionFunc
for child in context[-1].children:
context.append(child)
evaluation = criteria(context)
context.pop()
if evaluation:
return child
return None
# def _pick_child(context: List[Node], criteria: CriteriaType):
# """Returns the first child that meets the criteria."""
# if isinstance(criteria, int):
# try:
# return context[-1].children[criteria]
# except IndexError:
# return None
# elif isinstance(criteria, str):
# for child in context[-1].children:
# if child.tag_name == criteria:
# return child
# return None
# else: # assume criteria has type ConditionFunc
# for child in context[-1].children:
# context.append(child)
# evaluation = criteria(context)
# context.pop()
# if evaluation:
# return child
# return None
#######################################################################
......@@ -598,15 +599,16 @@ def flatten(context: List[Node], condition: Callable=is_anonymous, recursive: bo
node = context[-1]
if node.children:
new_result = [] # type: List[Node]
context.append(ZOMBIE_NODE)
for child in node.children:
context.append(child)
context[-1] = child
if child.children and condition(context):
if recursive:
flatten(context, condition, recursive)
new_result.extend(child.children)
else:
new_result.append(child)
context.pop()
context.pop()
node.result = tuple(new_result)
......
......@@ -192,18 +192,27 @@ A resonable workflow for developing the grammar proceeds like this:
errors easier.
4. Next, you should run the test script. Usually, some test will fail at
the first attempt. So you'll keep revising the EBNF-grammar, adjusting and adding test cases until all tests pass.
the first attempt. So you'll keep revising the EBNF-grammar, adjusting and
adding test cases until all tests pass.
5. Now it is time to try and compile the example documents. By this time the
test-script should have generated the compile-script, which you can be
called with the example documents. Don't worry too much about the output, yet. What is important at this stage is merely whether the parser can
called with the example documents. Don't worry too much about the output,
yet. What is important at this stage is merely whether the parser can
handle the examples or not. If not, further test cases and adjustments the
EBNF grammar will be needed - or revision of the examples in case you decide to use different syntactic constructs.
EBNF grammar will be needed - or revision of the examples in case you
decide to use different syntactic constructs.
If all examples can be parsed, you go back to step one and add further more complex examples, and continue to do so until you have the feeling that you DSL's grammar is rich enough for all intended application cases.
If all examples can be parsed, you go back to step one and add further more
complex examples, and continue to do so until you have the feeling that you
DSL's grammar is rich enough for all intended application cases.
Let's try this with the trivial demo example that comes with creating a new project with the "dhparser.py"-script. Now, you have already seen that the
"example.dsl"-document merely contains a simple sequence of words: "Life is but a walking shadow" Now, wouldn't it be nice, if we could end this sequence with a full stop to turn it into a proper sentence. So, open "examples.dsl" with a text editor and add a full stop::
Let's try this with the trivial demo example that comes with creating a new
project with the "dhparser.py"-script. Now, you have already seen that the
"example.dsl"-document merely contains a simple sequence of words: "Life is
but a walking shadow" Now, wouldn't it be nice, if we could end this sequence
with a full stop to turn it into a proper sentence. So, open "examples.dsl"
with a text editor and add a full stop::
Life is but a walking shadow.
......@@ -215,6 +224,12 @@ Now, try to compile "examples.dsl" with the compile-script::
Since the grammar, obviously, did not allow full stops so far, the parser
returns an error message. The error message is pretty self-explanatory in this
case. (Often, you will unfortunately find that the error message are somewhat
difficult to decipher. In particular, because it so happens that an error the parser complains about is just the consequence of an error made at an earlier location that the parser may not have been able to recognize as such. We will learn more about how to avoid such situations, later.) EOF is actually the name of a parser that captures the end of the file, thus "EOF"! But instead of the expected end of file an, as of now, unparsable construct, namely a full stop followed by a line feed, signified by "\n", was found.
difficult to decipher. In particular, because it so happens that an error the
parser complains about is just the consequence of an error made at an earlier
location that the parser may not have been able to recognize as such. We will
learn more about how to avoid such situations, later.) EOF is actually the
name of a parser that captures the end of the file, thus "EOF"! But instead of
the expected end of file an, as of now, unparsable construct, namely a full
stop followed by a line feed, signified by "\n", was found.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment