Commit 9dd25e2b authored by Jean-Matthieu Gallard's avatar Jean-Matthieu Gallard
Browse files

Merge branch 'toolkitUpdate' into 'master'

Toolkit update

See merge request exahype/ExaHyPE-Documentation!1
parents 975d5987 edaa1a71
......@@ -28,7 +28,7 @@ as well as the objectives at the EU's
\exahype\ is completely open source.
% \exahype\ is a collection of open source tools and mainly written in C, C++,
% FORTARN, Java, Matlab and Python.
% FORTARN, Matlab and Python.
\section*{Who should read this document}
......
......@@ -31,23 +31,12 @@ If you want to run \exahype\ on a distributed memory cluster, you have to have
MPI installed. \exahype\ uses only very basic MPI routines (as provided with
MPI 1.3, e.g.).
\item
ExaHyPE's development environment relies on Java (Java Runtime Environment JRE).
We provide some standalone JAR files typically built against
Java 1.8 or newer. You can however always build your own version of all
development tools (recommended) as long as you have a Java compiler (Java
Development Kit JDK).
%Again, \exahype\ uses only very basic Java, so most versions should do.
\item
ExaHyPE's default build environment uses GNU Make.
\item
If you want to use ExaHyPE's optimised compute kernels, you have to have Python 3 available
on your development system. Furthermore, you have to install
Intel's \texttt{libxsmm}\footnote{Libxsmm is open source: \url{https://github.com/hfp/libxsmm}}
and Python's module \texttt{Jinja2}\footnote{Jinja2 is open source: \url{http://jinja.pocoo.org/}}.
A local installation script is made available.
ExaHyPE's development environment relies on Python 3. All additional Python 3
dependencies can be downloaded using the git submodules and the provided script.
\end{itemize}
......@@ -75,6 +64,8 @@ If you clone the repository (Variant 2), you have to add Peano manually.
\subsection*{Variant 1: Download an \exahype\ release}
%\textcolor{red}{TODO Dominic: dead link, whole part seems not up to date}
Open a browser and go to \url{http://csdemo.ddns.net/hpcsoftware/exahype}.
Switch to the \texttt{Download} tab, where you should find a section \texttt{Download of Snapshots}.
Find the release of your choice and download it.
......@@ -150,16 +141,31 @@ It is guaranteed to be compatible with the ExaHyPE.
% \texttt{exahype.tar.gz}.
% You need this one as well as either \texttt{ExaHyPE.jar} or
% \texttt{ExaHyPE-toolkit.tar.gz}.
% We do not provide support for the jar file, i.e.~it might not be compatible with
% your Java version, so we recommend to get the zipped archive.
% Please unzip the files.
%\pagebreak
\section*{Obtaining the dependencies}
ExaHyPE's development environment relies on a few Python 3 dependencies. They are
all registered as git submodule to the project.
Peano is also registered as a git submodule
To obtain them, run the update script:
\begin{code}
> ./Submodules/updateSubmodules.sh
\end{code}
A few options are proposed and can be seen using the \texttt{-h} flag.
\section*{Finish the setup}
Once you have unzipped all the archives into a directory of your choice, you
should see something like
%\textcolor{red}{TODO Dominic: seems wrong}
\begin{code}
> cd mywellsuiteddirectory
> ls
......@@ -172,6 +178,8 @@ You might choose to maintain a different directory structure or rely on a
previous \peano\ or \exahype\ installation. In this case, you have to adopt pathes
in all of your \exahype\ scripts from hereon.
%\textcolor{red}{TODO Dominic: these are symlinks now, please update}
Please also check that your \texttt{Peano} subdirectory holds the Peano AMR
directories:
\begin{code}
......@@ -187,89 +195,19 @@ mandatory.
\section{Dry run of development tools}
{\bf Remark:} The jar file deployed with \exahype\ (in its root folder) might
not work on your system. If this is the case, you have to rebuild the jar first. See the next
Section \ref{section:download:retranslate-jar} for instructions. If you rebuild
the jar file, it typically is located at \texttt{xxxx}
\vspace{0.8cm}
\noindent
To check whether you are ready to program new applications with \exahype, type
in
\begin{code}
> java -jar ExaHyPE.jar
> ./Toolkit/toolkit.sh -h
\end{code}
\noindent
This should give you the following welcome message:
% on some platforms, the code environment does not accept
% this ASCII art due to the ` accent grave sign. Use
% verbatim instead.
%\begin{verbatim}
\begin{code}
================================
___ _ _ ___ ___
/ __|_ ____ _| || |_ _/ _ \ __|
| _|\ \ / _` | __ | || | _/ _|
\___/_\_\__,_|_||_|\_, |_| \___|
|__/
================================
www.exahype.eu
================================
The project has received funding from the European Unions
Horizon 2020 research and innovation programme under grant
agreement No 671698 (ExaHyPE). It is based upon the PDE
framework Peano (www.peano-framework.org).
ERROR: Please provide input file as argument
\end{code}
%\end{verbatim}
\noindent
If you don't see this message, there's a problem with the ExaHyPE development
toolkit.
In this case, retranslate it.
\section{Retranslating the \exahype\ development toolkit}
\label{section:download:retranslate-jar}
\exahype\ relies on a Java toolkit to free the developers from writing most of
the glue code.
The consortium can not provide jar files for various Java versions as the one
you might find on your system.
Instead of messing around with Java compatibilities, we
recommend to build your own toolkit for your target system.
For this, unzip the toolkit sources (if you do not work with a cloned git
repository)
\begin{code}
> mkdir MyToolkit
> mv ExaHyPE-toolkit.tar.gz MyToolkit
> cd MyToolkit
> tar -xzvf ExaHyPE-toolkit.tar.gz
\end{code}
and invoke
\begin{code}
> ./build.sh
\end{code}
\noindent
You obtain a new jar file \texttt{ExaHyPE.jar} in the local directory that you
should use from hereon.
\exahype's toolkit uses the compiler compiler frontend SableCC
(\href{http://www.sablecc.org}{sablecc.org}). If you also want to regenerate the front-end, you have
to provide SableCC to the makefile. Usually, this should not be necessary.
This should give you a description of the various toolkit options. If you encounter
an error, please make sure the submodules are downloaded.
%\section{Optional installation variants}d
......
......@@ -60,7 +60,7 @@ end exahype-project
\noindent
To prepare this example for the simulation, we have to generate all glue code
\begin{code}
> java -jar Toolkit/dist/ExaHyPE.jar --not-interactive \
> ./Toolkit/toolkit.sh \
Demonstrators/EulerFV/EulerFV.exahype
\end{code}
......
......@@ -114,11 +114,10 @@ Doxygen but may be quite useless without Peano documentation.
\item Your application. You might want to document this if it has grown big or
if you want to understand the generated C++ code class hierarchy.
\item The ExaHyPE toolbox, explained in detail in chapters \ref{chapter:workflow}
and appendix \ref{chapter:a-toolbox}. The toolbox is written in Java. It's code
is also documented and can render to standalone webpages using the \texttt{javadoc}
program. However, this documentation will not help you when programming ExaHyPE
in C++. Thus we don't go into detail about this part of ExaHyPE.
\item The ExaHyPE toolkit, explained in detail in chapters \ref{chapter:workflow}.
The toolkit is written in Python 3. It's code is also documented. However, this
documentation will not help you when programming ExaHyPE in C++. Thus we don't go
into detail about this part of ExaHyPE.
\item Further ExaHyPE code generators such as the Python3 code which generates the
optimized kernels, cf. section \ref{sec:optimized-kernels}. This is one of the
......
......@@ -32,12 +32,12 @@ An \exahype\ specification is a text file where you
specify exactly what you want to solve in which way.
This file\footnote{Some examples can be found in the Toolkit directory or
on the \exahype\ webpage. They have the extension \texttt{.exahype}.}
is handed over to our \exahype\ toolkit, a small Java application generating all
is handed over to our \exahype\ toolkit, a small python 3 application generating all
required code.
This code (also linking to all other required resources such as parameter
sets or input files) then is handed over to a compiler.
You end up with an executable that may run without the
Java toolkit or any additional sources from \exahype.
Depending on the option used, see \ref{chapter:a-toolkit}, you end up with an executable that
may run without the python toolkit or any additional sources from \exahype.
It however rereads the specification file again for input files or some
parameters, e.g.
We could have worked with two different files, a specification file used to
......@@ -94,7 +94,7 @@ setups.
We hand the file over to the toolkit
\begin{code}
java -jar ExaHyPE.jar Demonstrators/TrivialProject.exahype
./Toolkit/toolkit.sh Demonstrators/TrivialProject.exahype
\end{code}
\noindent
......
......@@ -148,7 +148,7 @@ allows you to alter the plotter's behaviour.
Once we are satisfied with the parameters in our \exahype\ specification file, we hand it over to the \exahype\ toolkit:
\begin{code}
java -jar <mypath>/ExaHyPE.jar Euler2d.exahype
./Toolkit/toolkit.sh Euler2d.exahype
\end{code}
......
......@@ -20,7 +20,7 @@ in the following:
domain up to a specification which kind of solvers will exist and which
numerical schemes they realise.
\item This specification file is handed over to the \exahype\ toolkit which is
a Java tool. It internally relies on Python scripts and invokes the libxsmm
a Python tool. It internally relies on Python scripts and invokes the libxsmm
generator driver as well. A local build of the libxsmm's code generation driver
is therefore a prerequisite for using optimised kernels.
\item The toolkit yields a couple of files (Makefile, glue
......@@ -179,7 +179,7 @@ of generic simulations.
\subsection{Templated specification files}
As the ExaHyPE specification files are described by an unique grammar which
can in principle only be \emph{exactly} parsed by the SableCC Java tookit, it
can in principle only be \emph{exactly} parsed by the Python tookit, it
is useful to understand specification files as common text files when it comes
to batch processing. Typical attemps are therefore replacement of striking
patterns such as \texttt{@VARIABLENAME@} in a key-value manner. Such a mechanism
......
\chapter{The \exahype\ toolbox}\label{chapter:a-toolbox}
\chapter{The \exahype\ toolkit}\label{chapter:a-toolkit}
\noindent
The \exahype\ toolbox is a small standalone Java program (a single
executable JAR file). We refer to it as \emph{toolbox} or \emph{toolkit}
or sometimes just \emph{code generator}. The toolkit acts as sole interface for
users, i.e.~non-developers. We decided to realise it in Java for several
reasons:
The \exahype\ toolkit is a small standalone Python program. The toolkit acts as
sole interface for users, i.e.~non-developers. We decided to realise it in Python
for several reasons:
\begin{enumerate}
\item We can realise lots of syntax checks without blowing up the C++ code.
\item We have to generate code fragments in \exahype. The order of the
methods chosen for example has an impact on kernel calls and mapping variants.
This is easy in Java.
\item We prefer not to have several interfaces. With a Java front-end, we can
This is easy in Python.
\item We prefer not to have several interfaces. With a Python front-end, we can
directly generate the configuration rather than parsing configurations again.
\end{enumerate}
\noindent
Note that by default the toolkit will be called at runtime to validate the
specification file. If you wish to avoid this behavior you can generate the glue
code with the \texttt{-s} option (see below).
\section{Dependencies}
\noindent
The toolkit itself requires Python 3.3 at least.
\noindent
It also use some python libraries that can be either directly installed through
pip or downloaded as a git submodule from ExaHyPE with the provided script
\begin{code}
./Submodules/updateSubmodules.sh
\end{code}
\section{Rebuilding the toolbox}
To (re-)build the toolbox, change into the toolkit directory and type in
\section{Running the toolkit}
\noindent
To run the toolkit, simply use the provided bash scrip
\begin{code}
./build.sh
./Toolkit/toolkit.sh
\end{code}
\noindent
The bash script is a wrapper over a sequence of make commands.
You can launch them directly by switching into the toolkit's \texttt{src}
directory and typing in
The bash script is a wrapper over a Python call. It will automatically check the
presence of all required dependencies.
\noindent
The toolkit itself can run with multiple options. The main ones being:
\begin{itemize}
\item \texttt{make clean}. Clean up the compiled and temporary files.
\item \texttt{make createParser}. Recreates the Java parser. \exahype's Java
parser relies on SableCC. The parser has to be regenerated if and only if you change
the grammar or you have to rebuild the tool without a previous build.
\item \texttt{make compile}. Translate the toolkit.
\item \texttt{make dist}. Pack the translated files into one jar file.
\item \texttt{-h}. Print the help and the list of available options.
\item \texttt{-v}. Run in verbose mode, the toolkit is quiet by default.
\item \texttt{-o}. Validate the given specification file. Do not generate any
ExaHyPE glue code.
\item \texttt{-j}. If the provided specification file isn't in JSON, the toolkit
will generate a translated one. Can be combined with \texttt{-o} to only translate
a specification file
\item \texttt{-s}. The toolkit won't be called at runtime but the specification
file used at runtime has to be in JSON. You can use \texttt{-jo} to (re)generate it
\end{itemize}
\section{Troubleshooting}
\label{section:appendix-toolkit:troubleshooting}
\begin{description}
\item [ The \texttt{build.sh} script seems to fail. ]
We recommend to follow the manual build steps as details in the section
above.
\item [ \exahype\ complains incompatible Java versions.]
We recommend that you rebuild the toolkit on every new machine.
\item [ \exahype\ complains about too old Java versions.]
The \exahype\ toolkit uses Java 8. We found it to be available on most
supercomputers and workstations though it often is not the default Java
version.
On your own machine, you switch the Java version with
\begin{code}
update-alternatives --config java
\end{code}
On supercomputers, you typically invoke a \texttt{module load} command.
\item [ Running the toolbox fails with messages alike 'Could not allocate
metaspace: 1073741824 bytes'.]
\exahype's toolbox jar tends to become rather big as it comprises lots of
templates and wrappers around other software. Usually, invoking Java with
increased heap provisions however resolves the issue:
\begin{code}
java -XX:MaxHeapSize=512m -jar Toolkit/dist/ExaHyPE.jar ...
\end{code}
\end{description}
% \noindent
% The \exahype\ toolbox is a small standalone Java program (a single
% executable JAR file). We refer to it as \emph{toolbox} or \emph{toolkit}
% or sometimes just \emph{code generator}. The toolkit acts as sole interface for
% users, i.e.~non-developers. We decided to realise it in Java for several
% reasons:
% \begin{enumerate}
% \item We can realise lots of syntax checks without blowing up the C++ code.
% \item We have to generate code fragments in \exahype. The order of the
% methods chosen for example has an impact on kernel calls and mapping variants.
% This is easy in Java.
% \item We prefer not to have several interfaces. With a Java front-end, we can
% directly generate the configuration rather than parsing configurations again.
% \end{enumerate}
% \section{Rebuilding the toolbox}
% To (re-)build the toolbox, change into the toolkit directory and type in
% \begin{code}
% ./build.sh
% \end{code}
% \noindent
% The bash script is a wrapper over a sequence of make commands.
% You can launch them directly by switching into the toolkit's \texttt{src}
% directory and typing in
% \begin{itemize}
% \item \texttt{make clean}. Clean up the compiled and temporary files.
% \item \texttt{make createParser}. Recreates the Java parser. \exahype's Java
% parser relies on SableCC. The parser has to be regenerated if and only if you change
% the grammar or you have to rebuild the tool without a previous build.
% \item \texttt{make compile}. Translate the toolkit.
% \item \texttt{make dist}. Pack the translated files into one jar file.
% \end{itemize}
% \section{Troubleshooting}
% \label{section:appendix-toolkit:troubleshooting}
% \begin{description}
% \item [ The \texttt{build.sh} script seems to fail. ]
% We recommend to follow the manual build steps as details in the section
% above.
% \item [ \exahype\ complains incompatible Java versions.]
% We recommend that you rebuild the toolkit on every new machine.
% \item [ \exahype\ complains about too old Java versions.]
% The \exahype\ toolkit uses Java 8. We found it to be available on most
% supercomputers and workstations though it often is not the default Java
% version.
% On your own machine, you switch the Java version with
% \begin{code}
% update-alternatives --config java
% \end{code}
% On supercomputers, you typically invoke a \texttt{module load} command.
% \item [ Running the toolbox fails with messages alike 'Could not allocate
% metaspace: 1073741824 bytes'.]
% \exahype's toolbox jar tends to become rather big as it comprises lots of
% templates and wrappers around other software. Usually, invoking Java with
% increased heap provisions however resolves the issue:
% \begin{code}
% java -XX:MaxHeapSize=512m -jar Toolkit/dist/ExaHyPE.jar ...
% \end{code}
% \end{description}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment