Skip to content
GitLab
Commit 63bdb434 authored by Olga Glogowska's avatar Olga Glogowska
Browse files

edited: Chapter 1: rearangement of subsections, initial description of... 

edited: Chapter 1: rearangement of subsections, initial description of installation steps, some TODOs to be discussed
parent 2b7cfba2
Hide whitespace changes
Inline Side-by-side
01_download.tex
View file @ 63bdb434
\chapter{Setup and Installation}
\label{chapter:download}
\section{Dependencies and prerequisites}
ExaHyPE comes along with the following dependencies:
\begin{itemize}
\item
\exahype\ source code is C++ code.
For sequential simulations, only a C++ compiler is required.
All examples from this guidebook run and have been tested with
newish GCC and Intel compilers.
The code uses few C++14 features, but for many older versions
enabling those features through \texttt{--std=c++0x} made the code
pass.
There are no further dependencies or libraries required.
\item
You may prefer to write parts of your \exahype\ code in Fortran. In this case,
you obviously need a Fortran besides a C++ compiler. \exahype\ itself does
not depend on Fortran.
\item
If you want \exahype\ to exploit your multi- or manycore computer, you have to
have Intel's TBB. it is open source and works with GCC and
Intel compilers. At least TBB 2017 is required.
\item
If you want to run \exahype\ on a distributed memory cluster, you have to have
MPI installed. \exahype\ uses only very basic MPI routines (as provided with
MPI 1.3, e.g.).
\item
ExaHyPE's default build environment uses GNU Make.
\item
ExaHyPE's development environment relies on Python 3. All additional Python 3
dependencies can be downloaded using the git submodules and the provided script.
\end{itemize}
%\section{Dependencies and prerequisites}
%ExaHyPE comes along with the following dependencies:
%\begin{itemize}
% \item
%\exahype\ source code is C++ code.
%For sequential simulations, only a C++ compiler is required.
%All examples from this guidebook run and have been tested with
%newish GCC and Intel compilers.
%The code uses few C++14 features, but for many older versions
%enabling those features through \texttt{--std=c++0x} made the code
%pass.
%There are no further dependencies or libraries required.
%
% \item
%You may prefer to write parts of your \exahype\ code in Fortran. In this case,
%you obviously need a Fortran besides a C++ compiler. \exahype\ itself does
%not depend on Fortran.
%
% \item
%If you want \exahype\ to exploit your multi- or manycore computer, you have to
%have Intel's TBB. it is open source and works with GCC and
%Intel compilers. At least TBB 2017 is required.
%
% \item
%If you want to run \exahype\ on a distributed memory cluster, you have to have
%MPI installed. \exahype\ uses only very basic MPI routines (as provided with
%MPI 1.3, e.g.).
%
% \item
%ExaHyPE's default build environment uses GNU Make.
%
% \item
%ExaHyPE's development environment relies on Python 3. All additional Python 3
%dependencies can be downloaded using the git submodules and the provided script.
%\end{itemize}
%
%\noindent
%The guidebook assumes that you use a Linux system. Members of the
%\exahype\ consortium and other users successfully use Windows and Mac systems.
%\exahype\ however focuses not on these platforms and thus no typical pitfalls
%are discussed here.
%The code itself is minimalistic, i.e.~in \exahype's basic form no further
%libraries are required.
\section{Prerequisites}
\noindent
The guidebook assumes that you use a Linux system. Members of the
\exahype\ consortium and other users successfully use Windows and Mac systems.
\exahype\ however focuses not on these platforms and thus no typical pitfalls
\exahype\ , however, does not focus on these platforms and thus no typical pitfalls
are discussed here.
The code itself is minimalistic, i.e.~in \exahype's basic form no further
libraries are required.
\subsection*{Development enviromnet: Python 3}
\section{Obtaining ExaHyPE}
ExaHyPE's development environment relies on Python 3. Python 3.3 or more is required. To check if and what version of Python 3 has been installed on your machine, you can type the following into your terminal:
\begin{code}
> python3 --version
\end{code}
You should see an output similar to the one shown below:
\begin{code}
> Python 3.7.3
\end{code}
Otherwise, you can install a minimal Python 3 package by running the following command:
\begin{code}
> sudo apt-get install python3
\end{code}
\exahype\ is available as source code only. We discuss several variants how to
obtain the code below.
\exahype\ is built on top of the AMR framework \peano\ .
\subsection*{Build enviromnet: GNU Make}
ExaHyPE's default build environment uses GNU Make. To check whether you have GNU Make installed on your machine, you can type the following into your terminal:
\begin{code}
> make --version
\end{code}
You should see an output similar to the one shown below:
\begin{code}
> GNU Make 4.2.1
> Built for x86_64-pc-linux-gnu
> Copyright (C) 1988-2016 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
\end{code}
Otherwise, you can install GNU Make by running the following command:
\begin{code}
> sudo apt-get install build-essential
\end{code}
This will also install other packages required for building from source, e.g. GCC (GNU Compiler Collection).
\subsection*{Compiler: C / C++}
\exahype\ source code is C++ code. For sequential simulations, only a C++ compiler is required.\\
All examples from this guidebook run and have been tested with newish GCC and Intel compilers. The code uses few C++14 features, so it is advised to use \texttt{gcc}/\texttt{g++} (5.0 or later) or \texttt{icpc} (2018 or later). For many older versions enabling those features through \texttt{--std=c++0x} or \texttt{--std=c++14} should make the code pass too, though. \\
If you wish to use Intel compiler, you need to install it from source. However, both \texttt{gcc} and \texttt{g++} should typically come as part of \texttt{build-essential} package.
To check if either of these has been installed on your machine, type the following into your terminal:
\begin{code}
> gcc --version
\end{code}
\begin{code}
> g++ --version
\end{code}
To install \texttt{gcc} or \texttt{g++} run:
\begin{code}
> sudo apt-get install gcc
\end{code}
\begin{code}
> sudo apt-get install g++
\end{code}
\noindent
\exahype\ itself does not depend on Fortran but you may prefer to write parts of your \exahype\ code in Fortran. In this case, you obviously need an additional Fortran compiler. Typically, \texttt{gfortran} comes with \texttt{build-essential} package. If you want an Intel \texttt{ifort}, you need to install from source.
\todo[inline]{TODO: Is actually only g++ needed and used by \exahype\ or is gcc also acceptable? Should gcc be deleted from description here? It should also compile C++ code, right? }
\subsection*{Shared memory support: TBB (or OpenMP)}
\exahype\ currently supports shared memory parallelisation through Intel’s Threading Building Blocks (TBB) or OpenMP. OpenMP support comes along with both Intel and GCC compilers. To check which version of OpenMP is supported by your GCC compiler you can type the following:
\begin{code}
> echo |cpp -fopenmp -dM |grep -i open
\end{code}
You should get otuput similar to the one shown below:
\begin{code}
#define _OPENMP 201511
\end{code}
the numbers following \texttt{\_OPENMP} variable are in the following format \texttt{yyyymm}, wehre \texttt{yyyy} represents the year and \texttt{mm} the mont of the release. You can go to \url{https://www.openmp.org/specifications/} to discover the mapping between the date and the actual version of the standard.\\
\noindent
We recommend using the TBB variant, though. The reason fot that is, that it is typically one step ahead of the OpenMP support. TBB is open source and works both with GCC and Intel compilers. At least TBB 2017 is required. You can clone TBB from the official repository by typing the following into your terminal:
\begin{code}
> cd path/to/where/you/want/to/install
> git clone https://github.com/intel/tbb.git
\end{code}
The above should result in creation of \texttt{tbb} directory under the given location. To build \texttt{TBB} you should use the top-level \texttt{Makefile}. Type the following into your terminal:
\begin{code}
> cd tbb
> make
\end{code}
For more information about how to tailor the build, please open \texttt{tbb/index.html} and go to \texttt{build directions} subsite. After successfull build, you need to modify your \texttt{bashrc} file:
\begin{code}
> gedit ~/.bashrc
\end{code}
by adding the following two lines:
\begin{code}
export TBB_INC="-I /path/to/where/you/installed/tbb/include"
export TBB_SHLIB="-L /path/to/where/you/installed/tbb/build/
linux_intel64_gcc_cc5.4.0_libc2.23_kernel4.15.0_release -ltbb"
\end{code}
and then hitting:
\begin{code}
> source ~/.bashrc
\end{code}
If later on (while trying to run your compiled \exahype\ application), you are still getting error similar to the one shown below:
\begin{code}
> ./ExaHyPE-MySolver: error while loading shared libraries:
libtbb.so.2: cannot open shared object file: No such file or directory
\end{code}
add an additional line to your \texttt{bashrc} file:
\begin{code}
export LD_LIBARARY_PATH=
"path/to/where/you/installed/tbb/build/
linux_intel64_gcc_cc5.4.0_libc2.23_kernel4.15.0_release ${LD_LIBARARY_PATH}"
\end{code}
and again hit:
\begin{code}
> source ~/.bashrc
\end{code}
Now the application should be able to run smoothly.
\todo[inline]{TODO: Is this possible to run \exahype without TBB (or OpenMP) at all? How to do this? When I deleted shared-memory part form EulerADERDG.exahype it would not compile. Although I did export SHAREDMEM=None, some Peano references to TBB were popping out after hitting make. Is it possible to use Peano without TBB? How to do this? Note: Peano 3 installed on my own from tar as I do not have access to repository yet. What needs to be done if we want to run shared-memory with OpenMP instead? Which variables need to be exported? Should this also be described here? Or should this whole configuration of bashrc for TBB be postponed to the section on shared-memory configuration instead? To keep it simple here and just make sure the people have insalled the prerequisites. Though, the EulerADERDG.exahype demonstrator uses shared-memory and/or distributed-memory already.}
\subsection*{Distributed memory support: MPI}
If you want to run \exahype\ on a distributed memory cluster, you have to have
MPI installed. \exahype\ uses only very basic MPI routines (as provided with e.g.
MPI 1.3). What you need to has installed on your machine is \texttt{MPICH}, i.e. a high-performance and widely portable implementation of the MPI. To check if it has been installed, you can type one of the following commands into your terminal:
\begin{code}
> mpichversion
\end{code}
\begin{code}
> mpiexec --version
\end{code}
\begin{code}
> mpirun --version
\end{code}
To install \texttt{MPICH} support, you can run:
\begin{code}
> sudo apt-get install mpich
\end{code}
\subsection*{Variant 1: Download an \exahype\ release}
\section{Obtaining \exahype\ }
\exahype\ is available as source code only. We discuss two variants of how to obtain the code below. The main difference between them, is in the way the dependencies of \exahype\ need to be handelled.
\subsection*{Variant 1: Download an \exahype\ release (highly recommended)}
Open a browser and go to \url{http://www.peano-framework.org}.
Here, click on \texttt{Request Repository Access} and fill
......@@ -65,8 +197,7 @@ out the application form. Click on \texttt{Register}. The application will
trigger a creation of a LRZ account linked to the \peano\ and \exahype\ projects.
Eventually, you will get reporter access to the software repositories via
\url{https://gitlab.lrz.de}. You will get notified by mail as soon
as this is the case. The process can take a few days.
as this is the case. The process can take a few days.
The \exahype\ repository can then be cloned via:
\begin{code}
> git clone https://gitlab.lrz.de/exahype/ExaHyPE-Engine.git
......@@ -75,8 +206,7 @@ or
\begin{code}
> git@gitlab.lrz.de:exahype/ExaHyPE-Engine.git
\end{code}
You can then install all dependencies using
You can then install all dependencies using:
\begin{code}
> ./Submodules/updateSubmodules.sh
\end{code}
......@@ -85,16 +215,39 @@ You can then install all dependencies using
\subsection*{Variant 2: Download an \exahype\ snapshot (not recommended)}
Go to \url{http://www.peano-framework.org/index.php/exahype/} and download a tar.gz file of a recent release. A
snapshot of \peano\ is included.
If this option is chosen all dependencies must be installed manually.
\section{Obtaining and configuring \exahype\ dependencies}
\section*{Obtaining the dependencies}
ExaHyPE's development environment relies on a few Python 3 dependencies. They are
all registered as git submodule to the project. \peano\ is also registered as a
git submodule
\exahype\ comes with the following dependencies:
To obtain them, run the update script:
\begin{itemize}
\item
\texttt{Peano}
\item
\texttt{LIBXSMM}
\item
\texttt{Python 3}
\begin{itemize}
\item
\texttt{jinja2}
\item
\texttt{jsonschema}
\item
\texttt{markupsafe}
\item
\texttt{attr}
\item
\texttt{pyrsistent}
\item
\texttt{six}
\end{itemize}
\end{itemize}
\subsection*{Variant 1: Use the provided script \texttt{updateSubmodules.sh} (highly recommended)}
All of \exahype\ dependencies listed above are registered as git submodules to the project.
To obtain these dependencies in a simple way, just run the provided script:
\begin{code}
> ./Submodules/updateSubmodules.sh
\end{code}
......@@ -114,13 +267,46 @@ required to create an SSH tunnel to \texttt{github.com} or
\texttt{gitlab.lrz.de} in order to clone repositories.
If the default port 12345 does not work for you, just change it in the script.
\subsection*{Variant 1: Install manually (not recommended)}
\subsubsection*{Peano}
\exahype\ is built on top of the AMR framework \peano\ .
In case you use \peano\ in several projects, you might want to skip the
download of the \peano\ submodule and instead add two symbolic links
to \peano's \texttt{peano} and \texttt{tarch} directory in the \texttt{Peano}
subdirectory of \exahype.
download of the \peano\ submodule and instead install it manually. If you do not have access to \peano's repository, you can download a snapshot from \url{here link}. In order to make \peano\ work with \exahype , you need to add two symbolic links to \peano's \texttt{peano} and \texttt{tarch} directories in the \texttt{Peano} subdirectory of \exahype.
You further need to create symbolic links to \peano's toolboxes
\texttt{mpibalancing}, \texttt{multiscalelinkedcell},
\texttt{sharedmemoryoracles}
\texttt{sharedmemoryoracles}. Apart from that you also need to create a link to \texttt{domaindecompositionanalysis.py} in \texttt{Toolbox} directory.
\todo[inline]{TODO: this has been tested already and works, but fill in with required code, i.e. how to download \peano\ without git access and create (symbolic or hard) links, etc. }
\subsubsection*{LIBXSMM}
\texttt{LIBXSSM} is used by \exahype's \texttt{CodeGeneretor} to perform efficient matrix-matrix multiplications. You can install it by following these steps. First clone the source code from the git repository to your desired install directory <my-path>:
\begin{code}
git clone -b release --single-branch https://github.com/hfp/libxsmm.git <my-path>/libxsmm
\end{code}
Compile the source code with:
\begin{code}
make realclean && make generator
\end{code}
In \texttt{ExaHyPE-Engine/CodeGenerator/codegenerator/configuration.py} put the correct path to the \texttt{libxsmm\_gemm\_generator}. Alternatively you can create appropriate symbolic links in the \texttt{Submodules} subdirectory or install this module directly in this directory. In that case, no chages to configuration file are required.
\todo[inline]{TODO: check if this works, actually follow the steps, for now it is a ``theory'' based on README files found in \exahype\ CodeGenerator directory and updateSubmodules.sh}
\subsubsection*{Python 3}
ExaHyPE’s development environment relies on a few Python 3 dependencies, which typically do not come along with the minimal install of \texttt{Python 3} package. If you decide to install these packages manually, you will need to follow the steps given below.
You can install \texttt{jsonschema} and \texttt{jinja2} using the following commands:
\begin{code}
pip3 install jsonschema
\end{code}
or
\begin{code}
apt-get install python-json-schema-validator
\end{code}
and
\begin{code}
pip3 install jinja2
\end{code}
Dependencies of \texttt{jinja2} and \texttt{jsonschema} are usually installed automatically when using \texttt{pip3}. Otherwise, you will also need to manually install the following modules: \texttt{markupsafe}, \texttt{attr}, \texttt{pyrsistent} and \texttt{six}.
Both \texttt{Toolkit} and \texttt{CodeGenerator} rely on these packages. Depeneding on where you have installed them, you need to update the paths in configuration files \texttt{Toolkit/exahype/toolkit/configuration.py}, \texttt{Toolkit/exahype/specfiles/configuration.py} and \texttt{CodeGenerator/codegenerator/configuration.py}. Alternatively you can create appropriate symbolic links to these packages in the \texttt{Submodules} subdirectory, or install the modules directly in this directory. In that case, no chages to configuration files are required.
\todo[inline]{TODO : check if this works, actually follow the steps, for now it is a ``theory'' based on README files found in \exahype Toolkit and CodeGenerator directories and updateSubmodules.sh}
\section{Dry run of development tools}
......@@ -136,5 +322,5 @@ in
\noindent
This should give you a description of the various toolkit options. If you encounter
an error, please make sure the submodules are downloaded.
an error, please make sure the submodules are downloaded and / or properlly linked with \exahype\ application.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment