Proposal for binary files handling, a new repository and a push service for JAR, TAR.GZ and PDFs
This is a short proposal how we could deal with the issue of peano.tar.gz
, ExaHyPE.jar
and guidebook.pdf
in the future.
Current status
- Peano is provided as tarball at https://sourceforge.net/projects/peano/files/ and copied into the ExaHyPE git repo by Tobias
- The JAR is recompiled from the ExaHyPE git repo and commited to the git repo
- The Guidebook PDF is not contained in the git repo but compiled and uploaded to http://www5.in.tum.de/exahype/guidebook.pdf, skipping the git repo
Disadvantages of current way to deal with it
- The git repository got huge: It already has >150MB in size which are mostly old copies of
peano.tar.gz
andExaHyPE.jar
. Every new installation needs to download all this. For comparison, 5 years of Peano development created a 2GB repository (SVN). A 1:1 conversion to git would require to download 2GB of old binaries everytime one wants to checkout the code. - Impossible to edit stuff in Peano
This issue combines two ideas who stand for their own:
Proposal A: How to deal with binaries in future
Currently, we do nightly builds. We should instead do builds on every push (ie. web hooks). Ie, everytime somebody pushes new commits to gitlab, a web hook starts up an ant build ...
invocation to create ExaHyPE.jar
as well as guidebook.pdf
and makes all these files available at a well-known address, for instance http://www5.in.tum.de/exahype/exahype.jar and http://www5.in.tum.de/exahype/guidebook.pdf.
This still does not solve the hazzle with Peano, but for the time being we should stick to https://sourceforge.net/projects/peano/files/latest/download?source=files or a wrapper similar to a git remote, ie a shell file which does a SVN checkout at a specific revision:
#!/bin/sh
# this script located at <ExaHyPE repository>/Code/Peano/
svn checkout -r 1234 svn://svn.code.sf.net/p/peano/code/trunk/src
where 1234
is the revision that corresponds to the local git commit (in the ExaHyPE repository) in the same manner as a git remote connects a specific version of a remote repository to the local git commit.
Proposal B: How to deal with the git repository in a time beyond binaries
After we got rid of the huge binaries, we can ask the question whether we can remove them from history. Actually, this is possible in git but changes the whole repository, as all the checksums from the old commits change. It is highly undesierable to do this at a repository which is cloned by so many people.
To avoid this, we could split the repository into two:
- A new fresh repository holding only source code, ie. the
Applications
,ExaHyPE core
andPeano integration links
. It will have a reasonable decent size. It could (or should) probably even hold the old but changed history without the binaries. - The old repository which from now on only contains the
Guidebook
and theTalks
folder which was not even there before the July 2016 codingweek. These two folders contain LaTeX files which involve the inclusion of pictures which also represent binary objects (PNG, PDF, etc.). This repository will keep increasing in size (>200MB ...) but it's no more required for everybody to download this stuff.
The two repositories could be located at https://gitlab.lrz.de/exahype/ExaHyPE-code and https://gitlab.lrz.de/exahype/ExaHyPE-docs. Especially, the docs
repository can already directly be mirrored to http://github.com/exahype.