mmh, hard to say what the reason is. I would like to look at your datasets. So, if you could mail me the third dataset I will have a look at this time sink.
Freie Universität Berlin
Institut für Informatik
Phone: +49 30 838 75137
Takustraße 9Algorithmic Bioinformatics
14195 BerlinRoom 020
I am currently observing some maybe undesired behavior using razers3, trunk version (checked out 29/08/2012, 9am, Berlin time).
Within a Rabema scenario the following is observed (impatient readers go straight to (3)):
1) 100,000 reads from Illumina GAII, single-end, 100Nt
../bin/rabema/razers3 -v -tc 32 -of sam -rr 100 -i 90 -m 1000000 -ds -o ./rabema/NA18507_1_1e5complex.sam ../ref_genomes/hg19.fa ../reads/NA18507_1_1e5complex.fastq
=> ran entirely and finished successfully.
2) 100,000 454-Titanium reads, ranging from 50Nt up to 3575
../bin/rabema/razers3 -v -tc 32 -of sam -rr 100 -i 90 -m 1000000 -ds -o ./rabema/NA12878_1e5complex.sam ../ref_genomes/hg19.fa ../reads/NA12878_1e5complex.fastq
=> didn't yet finish, but steady progress is observed, while mem-consumption seems reasonable (below 20Gb).
3) 100,000 IonTorrent reads, ranging from 5Nt up to 387(less than 100 reads are actually shorter than 50 nukes)
../bin/rabema/razers3 -v -tc 32 -of sam -rr 100 -i 90 -m 1000000 -ds -o ./rabema/NS12911_1e5complex.sam ../ref_genomes/hg19.fa ../reads/NS12911_1e5complex.fastq
=> this didn't finish and no progress is observed any more.
### the according STDERR-output is:
Genome file: ../ref_genomes/hg19.fa
Read file: ../reads/NS12911_1e5complex.fastq
Compute forward matches: YES
Compute reverse matches: YES
Allow Indels: YES
Error rate: 0.1
Pigeonhole mode with overlap: 0
Repeat threshold: 1000
Overabundance threshold: 1
Program PID: 29923
25014636 bps of 100000 reads loaded.
Loading reads took 0.507097 seconds
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29923 uappelt 25 0 136g 97g 1928 R 3197.9 77.3 141:03.96 razers3
29931 uappelt 16 0 11544 1772 780 R 1.0 0.0 0:02.60 top
### My according output is: *DOOOOH*[###EOF###]
So sample 3 kind of skrewed the machine. The poor thing is close to swapping and CPUs are under heavy loads, while progress didn't move a single period further (within minutes). I guess, extremely short reads (below 50 nukes) are the culprits to blame, since
this phenomenon doesn't occur with longer reads in samples 1 & 2. Still, less than 100 reads are in fact shorter than 50 nukes. And even a million reference-genome locations that were asked to be reported per read shouldn't cause this, should it.
Is there an easy way to prevent this behavior? Filtering for short reads would certainly be an option. Couldn't, however, long reads entirely lying within repetitive elements (e.g. SINES or LINES) cause similar malloc-behavior? This case couldn't be taken care
of so easily from outside the read mapper, could it.
Let me know, if you want reads of sample 3 to be emailed somewhere (gzipped file size: 22,558 KB)
seqan-dev mailing list email@example.com