Re: [Seqan-dev] Seqan 1.4 : (OpenMS 1.11 packaging in Debian)

[Filippo Rusconi <listes.rusconi@laposte.net>]

Greetings Hannes, and Fellow Developers,

[[ CC seqan-dev@lists.fu-berlin.de as this discussion is relevant to
the seqan developers, who kindly offered to try to find a solution to
the problems that have arisen for Debian packaging after the switch to
svn-based software releases has been implemented ]]

As a follow-up of my previous ramblings about packaging seqan for
Debian, which is required since OpenMS depends on it, here are my
tests.

I want to remind you of some points:

seqan was packaged for Debian in the form of two binary packages:
      - seqan-dev
      - sequan-apps

Since the most recent versions of seqan, the software cannot be
packaged in Debian because the source release mechanisms have
changed. Specifically, there are no more source tarballs for each new
version, upon which to make packages for Debian.

In lieu of the tarballs, now the developers are required to download
full svn tagged branches that are, of course, way too large for making
a source tarball by simply compressing the resulting directories.

Because Debian uses--in the package creation process--a source tarball
that is versioned and absolutely immutable (that is, with an immutable
checksum), we cannot package seqan anymore.

I have followed the descriptions of Manuel (see thread
https://lists.fu-berlin.de/htdig/seqan-dev/2013-August/msg00022.html)
to build both the library and the apps subpackages. To do that, I have
first "purified" the directory in which the svn branch was checked
out.

Here is a transcript of the my source tree "purifying" commands,
interspersed with commands to monitor the size of the tree after each
removal of unused stuff:


$ svn co http://svn.seqan.de/seqan/tags/seqan-1.4.1 seqan-1.4.1

$ du -sh seqan-1.4.1
478M    seqan-1.4.1

./core/tests$ du -sh .svn
225M    .svn
# We'll remove that .svn directory later


$ find -name "tests" -type d | xargs rm -rf
./core/apps/sak/tests
./core/apps/mason/tests
./core/apps/seqan_tcoffee/tests
./core/apps/razers/tests
./core/apps/rabema/tests
./core/apps/stellar/tests
./core/apps/razers2/tests
./core/apps/pair_align/tests
./core/apps/snp_store/tests
./core/apps/dfi/tests
./core/apps/splazers/tests
./core/apps/micro_razers/tests
./core/apps/tree_recon/tests
./extras/tests
./extras/apps/razers3/tests
./extras/apps/insegt/tests
./extras/apps/gustaf/tests
./extras/apps/searchjoin/tests
./extras/apps/masai/tests
./extras/apps/variant_comp/tests
./extras/apps/sgip/tests
./extras/apps/breakpoint_calculator/tests
./misc/seqan_instrumentation/bin/classes/simplejson/tests

$ du -sh ../seqan-1.4.1
319M    ../seqan-1.4.1

$ rm -rf extras/

(We'll see later that, for the time being, we need this directory to
build non-extra stuff)

$ du -sh ../seqan-1.4.1
311M    ../seqan-1.4.1

$ rm -rf .svn
$ du -sh ../seqan-1.4.1
87M     ../seqan-1.4.1

# Good, that seems reasonable, now. When compressed, that will make a
source tarball something like 30 M of size, which is fine.

# Remove the corresponding directory from the CMakeLists main CMake file.
# Delete lines
-> message (STATUS "Configuring extras")
-> add_subdirectory (extras)

# Now test the build

Ok, start with the apps
=======================

cd ..
mkdir -p seqan-1.4.1-build/release
cd seqan-1.4.1-build/release
cmake ../../seqan-1.4.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../seqan-1.4.1-install -DSEQAN_BUILD_SYSTEM=SEQAN_RELEASE_APPS

$ make
Scanning dependencies of target dfi
[ 75%] Building CXX object core/apps/dfi/CMakeFiles/dfi.dir/dfi.cpp.o
/home/rusconi/devel/packaging/seqan/seqan-1.4.1/core/apps/dfi/dfi.cpp:29:45: fatal error: ../../extras/include/seqan/math.h: No such file or directory
 #include <../../extras/include/seqan/math.h>
                                             ^
compilation terminated.

# The point seems to be that the core/apps stuff needs headers that
  are located in extras/include/seqan. That' odd. How core stuff needs
  something in extras ?

By not removing the extras directory, the build went fine up to the
end. So, I decided to leave that extras directory in the source tree.


OK, now, to the library stuff.
==============================

$ cd ..
$ mkdir library
$ cd library
$ cmake ../../seqan-1.4.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../seqan-lib-1.4.1-install -DSEQAN_BUILD_SYSTEM=SEQAN_RELEASE_LIBRARY
$ make docs
$ make install

This seems to end-up with proper installation of the library
stuff. And I can see seq_io.h installed correctly, which is fine,
since this is the file that was missing during the OpenMS build with a
previous version of seqan.


Ramblings
=========

So, at this time I would say the following:

The size of the tarball is 

-rw-r--r--  1 rusconi rusconi  30M Sep 30 10:28 seqan-1.4.1-cleaned.tar.gz

which is perfectly right for us.

How could we automate this tarball creation step, that amounts to
cleaning up the build-unused stuff that is in the subversion tag
checkout ?

I suggest you would put a simple script like what I have done above in
the source tree and that you run it ONCE on the subversion branch
during a release, such as to produce a single versioned source
tarball.

For example, running this script against the 1.4.1 tag would afford a
single tar.gz file named:

seqan-1.4.1.tar.gz

That source tarball would then be made available on the downloads
page, maybe with some explanation of how it differs from the
corresponding tag checkout out of the svn repository.

How about such possibility ? I feel like this should work fine in the
long run, if the script is amended when the source tree changes
significantly. We, Debian Developers, would anyhow provide feedback
upon using the source tarball for the creation of the packages.

Please, let me know of your opinions on the matter above,

with my best regards,

Filippo Rusconi

-- 
Filippo Rusconi, PhD - public crypto key C78F687C @ pgp.mit.edu
Researcher at CNRS and Debian Developer

my massXpert software: http://www.massxpert.org