Command line: /home/ec2-user/PGTools/SPAdes/bin/spades.py -o /home/ec2-user/PGScratch/Varnish/SPAdes_bp8 -t 16 --pe1-1 /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq --pe1-2 /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq --pe1-fr --meta System information: SPAdes version: 3.9.0 Python version: 2.7.12 OS: Linux-4.4.11-23.53.amzn1.x86_64-x86_64-with-glibc2.2.5 Output dir: /home/ec2-user/PGScratch/Varnish/SPAdes_bp8 Mode: read error correction and assembling Debug mode is turned OFF Dataset parameters: Metagenomic mode Reads: Library number: 1, library type: paired-end orientation: fr left reads: ['/home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq'] right reads: ['/home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq'] interlaced reads: not specified single reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed (with gzip) Assembly parameters: k: [21, 33, 55] Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/tmp Threads: 16 Memory limit (in Gb): 240 ======= SPAdes pipeline started. Log can be found here: /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/spades.log ===== Read error correction started. == Running read error correction tool: /home/ec2-user/PGTools/SPAdes/bin/hammer /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/corrected/configs/config.info 0:00:00.000 4M / 4M INFO General (main.cpp : 83) Starting BayesHammer, built from N/A, git revision N/A 0:00:00.000 4M / 4M INFO General (main.cpp : 84) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/corrected/configs/config.info 0:00:00.000 4M / 4M INFO General (memory_limit.hpp : 47) Memory limit set to 240 Gb 0:00:00.000 4M / 4M INFO General (main.cpp : 93) Trying to determine PHRED offset 0:00:00.014 4M / 4M INFO General (main.cpp : 99) Determined value is 33 0:00:00.014 4M / 4M INFO General (hammer_tools.cpp : 36) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.014 4M / 4M INFO General (main.cpp : 120) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.014 4M / 4M INFO K-mer Counting (kmer_data.cpp : 280) Estimating k-mer count 0:00:00.156 260M / 260M INFO K-mer Counting (kmer_data.cpp : 285) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq 0:00:11.456 320M / 320M INFO K-mer Counting (kmer_data.cpp : 294) Processed 3147817 reads 0:00:11.456 320M / 320M INFO K-mer Counting (kmer_data.cpp : 285) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq 0:00:19.627 320M / 320M INFO K-mer Counting (kmer_data.cpp : 294) Processed 6295634 reads 0:00:19.628 320M / 320M INFO K-mer Counting (kmer_data.cpp : 299) Total 6295634 reads processed 0:00:21.521 320M / 320M INFO K-mer Counting (kmer_data.cpp : 306) Estimated 309890571 distinct kmers 0:00:21.535 64M / 320M INFO K-mer Counting (kmer_data.cpp : 311) Filtering singleton k-mers 0:00:21.951 952M / 952M INFO K-mer Counting (kmer_data.cpp : 317) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq 0:00:40.953 952M / 952M INFO K-mer Counting (kmer_data.cpp : 326) Processed 3147817 reads 0:00:40.953 952M / 952M INFO K-mer Counting (kmer_data.cpp : 317) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq 0:01:00.022 952M / 952M INFO K-mer Counting (kmer_data.cpp : 326) Processed 6295634 reads 0:01:00.022 952M / 952M INFO K-mer Counting (kmer_data.cpp : 331) Total 6295634 reads processed 0:01:00.022 952M / 952M INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:01:00.022 952M / 952M INFO K-mer Splitting (kmer_data.cpp : 91) Splitting kmer instances into 256 buckets. This might take a while. 0:01:00.023 952M / 952M INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:01:00.023 952M / 952M INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.98063 Gb 0:01:00.023 952M / 952M INFO General (kmer_index_builder.hpp : 113) Using cell size of 262144 0:01:00.057 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 98) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq 0:01:03.680 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 649141 reads 0:01:06.946 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 1241256 reads 0:01:10.642 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 1918068 reads 0:01:14.860 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 2669443 reads 0:01:17.516 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 3147817 reads 0:01:17.516 9G / 9G INFO K-mer Splitting (kmer_data.cpp : 98) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq 0:01:29.470 9G / 10G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 5361718 reads 0:01:34.607 9G / 10G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 6295634 reads 0:01:34.608 9G / 10G INFO K-mer Splitting (kmer_data.cpp : 113) Total 6295634 reads processed 0:01:34.608 9G / 10G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:01:50.842 9G / 10G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 113458062 kmers in total. 0:01:50.842 9G / 10G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:01:51.660 9G / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:01:55.657 9G / 12G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:02:26.786 9G / 12G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 37068984 bytes occupied (2.61376 bits per kmer). 0:02:27.188 64M / 12G INFO K-mer Counting (kmer_data.cpp : 357) Arranging kmers in hash map order 0:02:32.067 1G / 12G INFO General (main.cpp : 155) Clustering Hamming graph. 0:04:17.986 1G / 12G INFO General (main.cpp : 162) Extracting clusters 0:04:46.672 1G / 12G INFO General (main.cpp : 174) Clustering done. Total clusters: 94757910 0:04:46.713 932M / 12G INFO K-mer Counting (kmer_data.cpp : 379) Collecting K-mer information, this takes a while. 0:04:47.869 3G / 12G INFO K-mer Counting (kmer_data.cpp : 385) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq 0:05:28.577 3G / 12G INFO K-mer Counting (kmer_data.cpp : 385) Processing /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq 0:06:09.047 3G / 12G INFO K-mer Counting (kmer_data.cpp : 392) Collection done, postprocessing. 0:06:09.457 3G / 12G INFO K-mer Counting (kmer_data.cpp : 406) There are 113458062 kmers in total. Among them 2148816 (1.89393%) are singletons. 0:06:09.457 3G / 12G INFO General (main.cpp : 180) Subclustering Hamming graph 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 649) Subclustering done. Total 512 non-read kmers were generated. 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 650) Subclustering statistics: 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 651) Total singleton hamming clusters: 82721850. Among them 74098600 (89.5756%) are good 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 652) Total singleton subclusters: 861167. Among them 859246 (99.7769%) are good 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 653) Total non-singleton subcluster centers: 12435730. Among them 11553606 (92.9065%) are good 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 654) Average size of non-trivial subcluster: 2.4718 kmers 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 655) Average number of sub-clusters per non-singleton cluster: 1.10475 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 656) Total solid k-mers: 86511452 0:09:38.157 3G / 12G INFO Hamming Subclustering (kmer_cluster.cpp : 657) Substitution probabilities: [4,4]((0.951352,0.0134965,0.0165169,0.0186348),(0.0244393,0.940944,0.0207137,0.0139025),(0.0131248,0.0198756,0.941857,0.0251426),(0.0179778,0.0151301,0.0123255,0.954567)) 0:09:38.272 3G / 12G INFO General (main.cpp : 185) Finished clustering. 0:09:38.272 3G / 12G INFO General (main.cpp : 204) Starting solid k-mers expansion in 16 threads. 0:10:08.563 3G / 12G INFO General (main.cpp : 225) Solid k-mers iteration 0 produced 3096608 new k-mers. 0:10:38.417 3G / 12G INFO General (main.cpp : 225) Solid k-mers iteration 1 produced 55301 new k-mers. 0:11:08.222 3G / 12G INFO General (main.cpp : 225) Solid k-mers iteration 2 produced 2142 new k-mers. 0:11:38.053 3G / 12G INFO General (main.cpp : 225) Solid k-mers iteration 3 produced 148 new k-mers. 0:12:07.871 3G / 12G INFO General (main.cpp : 225) Solid k-mers iteration 4 produced 9 new k-mers. 0:12:07.871 3G / 12G INFO General (main.cpp : 229) Solid k-mers finalized 0:12:07.871 3G / 12G INFO General (hammer_tools.cpp : 211) Starting read correction in 16 threads. 0:12:07.871 3G / 12G INFO General (hammer_tools.cpp : 222) Correcting pair of reads: /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R1_out-adapter_out-quality_out-bbnorm.fastq and /home/ec2-user/PGScratch/Varnish/qc_bp8/normalization/varnish_bp8_shotgun_AD004_R2_out-adapter_out-quality_out-bbnorm.fastq 0:12:13.032 4G / 12G INFO General (hammer_tools.cpp : 166) Prepared batch 0 of 1600000 reads. 0:12:48.846 5G / 12G INFO General (hammer_tools.cpp : 175) Processed batch 0 0:12:52.223 5G / 12G INFO General (hammer_tools.cpp : 185) Written batch 0 0:12:56.948 5G / 12G INFO General (hammer_tools.cpp : 166) Prepared batch 1 of 1547817 reads. 0:13:34.036 5G / 12G INFO General (hammer_tools.cpp : 175) Processed batch 1 0:13:37.921 5G / 12G INFO General (hammer_tools.cpp : 185) Written batch 1 0:13:39.436 3G / 12G INFO General (hammer_tools.cpp : 270) Correction done. Changed 6667898 bases in 2345550 reads. 0:13:39.436 3G / 12G INFO General (hammer_tools.cpp : 271) Failed to correct 1090 bases out of 868533942. 0:13:39.628 64M / 12G INFO General (main.cpp : 262) Saving corrected dataset description to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/corrected/corrected.yaml 0:13:39.629 64M / 12G INFO General (main.cpp : 269) All done. Exiting. == Compressing corrected reads (with gzip) == Dataset description file was created: /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/corrected/corrected.yaml ===== Read error correction finished. ===== Assembling started. == Running assembler: K21 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K21/configs/config.info 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K21/configs/mda_mode.info 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K21/configs/meta_mode.info 0:00:00.000 4M / 4M INFO General (memory_limit.hpp : 47) Memory limit set to 240 Gb 0:00:00.000 4M / 4M INFO General (main.cpp : 88) Starting SPAdes, built from N/A, git revision N/A 0:00:00.000 4M / 4M INFO General (main.cpp : 89) Maximum k-mer length: 128 0:00:00.000 4M / 4M INFO General (main.cpp : 90) Assembling dataset (/home/ec2-user/PGScratch/Varnish/SPAdes_bp8/dataset.info) with K=21 0:00:00.000 4M / 4M INFO General (launch.hpp : 29) SPAdes started 0:00:00.000 4M / 4M INFO General (launch.hpp : 36) Starting from stage: construction 0:00:00.000 4M / 4M INFO General (launch.hpp : 39) Two-step RR enabled: 0 0:00:00.000 4M / 4M INFO StageManager (stage.cpp : 126) STAGE == Construction 0:00:00.000 4M / 4M INFO General (read_converter.hpp : 78) Converting reads to binary format for library #0 (takes a while) 0:00:00.000 4M / 4M INFO General (read_converter.hpp : 79) Converting paired reads 0:00:00.180 80M / 80M INFO General (binary_converter.hpp : 139) 16384 reads processed 0:00:00.315 96M / 96M INFO General (binary_converter.hpp : 139) 32768 reads processed 0:00:00.584 124M / 124M INFO General (binary_converter.hpp : 139) 65536 reads processed 0:00:01.115 180M / 180M INFO General (binary_converter.hpp : 139) 131072 reads processed 0:00:02.173 296M / 296M INFO General (binary_converter.hpp : 139) 262144 reads processed 0:00:04.286 520M / 520M INFO General (binary_converter.hpp : 139) 524288 reads processed 0:00:09.490 872M / 872M INFO General (binary_converter.hpp : 139) 1048576 reads processed 0:00:18.800 880M / 880M INFO General (binary_converter.hpp : 139) 2097152 reads processed 0:00:29.006 416M / 888M INFO General (binary_converter.hpp : 159) 3108625 reads written 0:00:29.473 4M / 888M INFO General (read_converter.hpp : 88) Converting single reads 0:00:29.631 140M / 888M INFO General (binary_converter.hpp : 139) 16384 reads processed 0:00:29.696 144M / 888M INFO General (binary_converter.hpp : 139) 32768 reads processed 0:00:29.770 148M / 888M INFO General (binary_converter.hpp : 159) 38749 reads written 0:00:29.794 4M / 888M INFO General (graph_construction.hpp : 120) Constructing DeBruijn graph for k=21 0:00:29.794 4M / 888M INFO General (kmer_splitters.hpp : 130) Splitting kmer instances into 256 buckets. This might take a while. 0:00:29.794 4M / 888M INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:00:29.794 4M / 888M INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.99992 Gb 0:00:29.794 4M / 888M INFO General (kmer_index_builder.hpp : 113) Using cell size of 262144 0:00:31.987 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 2612325 reads 0:00:34.130 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 5127005 reads 0:00:36.331 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 7784311 reads 0:00:39.914 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 11706866 reads 0:00:40.690 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 12511998 reads 0:00:40.690 8G / 9G INFO General (kmer_splitters.hpp : 160) Adding contigs from previous K 0:00:40.690 8G / 9G INFO General (kmer_splitters.hpp : 171) Used 12511998 reads. Maximum read length 155 0:00:40.690 8G / 9G INFO General (kmer_splitters.hpp : 172) Average read length 137.972 0:00:40.690 8G / 9G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:00:47.759 8G / 9G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 88088314 kmers in total. 0:00:47.759 8G / 9G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:00:48.392 8G / 9G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:00:48.392 8G / 9G INFO General (kmer_splitters.hpp : 282) Splitting kmer instances into 256 buckets. This might take a while. 0:00:48.392 8G / 9G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:00:48.392 8G / 9G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.8151 Gb 0:00:48.392 8G / 9G INFO General (kmer_index_builder.hpp : 113) Using cell size of 262144 0:00:54.131 17G / 17G INFO General (kmer_splitters.hpp : 301) Processed 88088314 kmers 0:00:54.131 17G / 17G INFO General (kmer_splitters.hpp : 306) Used 88088314 kmers. 0:00:54.160 17G / 17G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:00:54.396 17G / 17G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 87331638 kmers in total. 0:00:54.396 17G / 17G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:00:55.016 17G / 17G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:00:58.208 17G / 19G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:01:31.238 17G / 19G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 28533016 bytes occupied (2.61376 bits per kmer). 0:01:31.331 17G / 19G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 85) Building k-mer extensions from k+1-mers 0:01:37.732 17G / 19G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 89) Building k-mer extensions from k+1-mers finished. 0:01:38.124 148M / 19G INFO Early tip clipping (early_simplification.hpp : 182) Early tip clipping 0:01:50.273 148M / 19G INFO Early tip clipping (early_simplification.hpp : 185) 21593101 22-mers were removed by early tip clipper 0:01:50.273 148M / 19G INFO General (graph_construction.hpp : 136) Condensing graph 0:01:50.273 148M / 19G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 330) Extracting unbranching paths 0:02:31.170 644M / 19G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 344) Extracting unbranching paths finished. 5268657 sequences extracted 0:02:35.308 644M / 19G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 304) Collecting perfect loops 0:02:55.388 644M / 19G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 319) Collecting perfect loops finished. 74 loops collected 0:02:57.109 1G / 19G INFO General (graph_construction.hpp : 141) Building index with from graph 0:02:57.110 1G / 19G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:02:57.110 1G / 19G INFO General (kmer_splitters.hpp : 213) Splitting kmer instances into 16 buckets. This might take a while. 0:02:57.110 1G / 19G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:02:57.110 1G / 19G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 79.6094 Gb 0:02:57.110 1G / 19G INFO General (kmer_index_builder.hpp : 113) Using cell size of 4194304 0:03:05.248 1G / 19G INFO General (kmer_splitters.hpp : 224) Processed 5268731 edges 0:03:05.248 1G / 19G INFO General (kmer_splitters.hpp : 229) Used 5268731 sequences. 0:03:05.248 1G / 19G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:03:06.390 1G / 19G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 66495213 kmers in total. 0:03:06.390 1G / 19G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:03:06.832 1G / 19G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:03:28.209 1G / 19G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:03:46.978 1G / 19G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 21725336 bytes occupied (2.61376 bits per kmer). 0:03:47.708 2G / 19G INFO General (edge_index_builders.hpp : 27) Collecting k-mer coverage information from graph, this takes a while. 0:03:52.586 2G / 19G INFO General (edge_index.hpp : 91) Index refilled 0:03:52.789 2G / 19G INFO General (graph_construction.hpp : 173) Filling coverage index 0:03:52.789 2G / 19G INFO General (edge_index_builders.hpp : 105) Collecting k-mer coverage information from reads, this takes a while. 0:04:19.284 2G / 19G INFO General (graph_construction.hpp : 175) Filling coverage and flanking coverage from index 0:04:34.864 2G / 19G INFO General (construction.cpp : 30) Figured out: read length = 155 0:04:34.864 2G / 19G INFO StageManager (stage.cpp : 126) STAGE == EC Threshold Finding 0:04:38.070 2G / 19G INFO ThresholdFinder (ec_threshold_finder.hpp : 114) Bucket size: 8 0:04:38.070 2G / 19G INFO General (genomic_info_filler.cpp : 104) Average edge coverage: 10.5825 0:04:38.070 2G / 19G INFO General (genomic_info_filler.cpp : 105) Graph threshold: 14 0:04:42.561 2G / 19G INFO General (genomic_info_filler.cpp : 147) EC coverage threshold value was calculated as 10.5825 0:04:42.561 2G / 19G INFO General (genomic_info_filler.cpp : 148) Trusted kmer low bound: 0 0:04:42.561 2G / 19G INFO StageManager (stage.cpp : 126) STAGE == Simplification 0:04:42.633 1G / 19G INFO General (simplification.cpp : 349) Graph simplification started 0:04:42.633 1G / 19G INFO General (simplification.cpp : 98) PROCEDURE == InitialCleaning 0:04:42.633 1G / 19G INFO General (graph_simplification.hpp : 577) Removing isolated edges 0:04:42.633 1G / 19G INFO General (graph_simplification.hpp : 578) All isolated edges shorter than 155 will be removed 0:04:42.633 1G / 19G INFO General (graph_simplification.hpp : 579) Also isolated edges shorter than 0 and coverage smaller than 0 will be removed 0:04:42.633 1G / 19G INFO General (graph_simplification.hpp : 850) Flanking coverage based disconnection disabled 0:04:42.633 1G / 19G INFO General (simplification.cpp : 326) Running Self conjugate edge remover 0:04:42.822 1G / 19G INFO General (simplification.cpp : 326) Running Initial isolated edge remover 0:04:43.227 1G / 19G INFO General (simplification.cpp : 326) Running Initial tip clipper 0:04:45.213 1G / 19G INFO General (simplification.cpp : 326) Running Initial ec remover 0:04:45.496 1G / 19G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:04:45.496 1G / 19G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 1 0:04:45.496 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:04:46.165 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:06:22.499 1G / 19G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:06:23.429 1G / 19G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 2 0:06:23.429 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:06:23.930 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:06:24.269 1G / 19G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:06:45.505 1G / 19G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 3 0:06:45.505 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:06:46.082 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:06:58.576 1G / 19G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:06.520 1G / 19G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 4 0:07:06.520 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:06.664 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:11.774 1G / 19G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:11.775 1G / 19G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 5 0:07:11.775 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:11.783 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:11.792 1G / 19G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:11.792 1G / 19G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 6 0:07:11.792 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:11.792 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:11.792 1G / 19G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:11.792 1G / 19G INFO General (simplification.cpp : 213) PROCEDURE == Post simplification 0:07:11.792 1G / 19G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:07:11.792 1G / 19G INFO General (simplification.cpp : 268) Iteration 0 0:07:11.792 1G / 19G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:23.452 1G / 19G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:25.077 1G / 19G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:25.077 1G / 19G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:25.545 1G / 19G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:27.565 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:27.946 1G / 19G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:28.283 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:30.278 1G / 19G INFO General (simplification.cpp : 268) Iteration 1 0:07:30.278 1G / 19G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:33.953 1G / 19G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:34.611 1G / 19G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:34.611 1G / 19G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:34.956 1G / 19G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:36.194 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:36.219 1G / 19G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:36.222 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:36.297 1G / 19G INFO General (simplification.cpp : 268) Iteration 2 0:07:36.297 1G / 19G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:39.614 1G / 19G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:40.229 1G / 19G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:40.229 1G / 19G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:40.563 1G / 19G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:41.785 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:41.786 1G / 19G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:41.786 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:41.808 1G / 19G INFO General (simplification.cpp : 268) Iteration 3 0:07:41.808 1G / 19G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:45.069 1G / 19G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:45.671 1G / 19G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:45.672 1G / 19G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:45.998 1G / 19G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:47.218 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:47.218 1G / 19G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:47.218 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:47.219 1G / 19G INFO General (simplification.cpp : 268) Iteration 4 0:07:47.219 1G / 19G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:50.484 1G / 19G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:51.089 1G / 19G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:51.089 1G / 19G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:51.428 1G / 19G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:52.645 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:52.645 1G / 19G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:52.645 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:52.646 1G / 19G INFO General (simplification.cpp : 268) Iteration 5 0:07:52.646 1G / 19G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:55.925 1G / 19G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:56.532 1G / 19G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:56.532 1G / 19G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:56.872 1G / 19G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:58.089 1G / 19G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:58.089 1G / 19G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:58.089 1G / 19G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:58.089 1G / 19G INFO General (simplification.cpp : 296) Disrupting self-conjugate edges 0:07:58.389 1G / 19G INFO StageManager (stage.cpp : 126) STAGE == Simplification Cleanup 0:07:58.389 1G / 19G INFO General (graph_simplification.hpp : 577) Removing isolated edges 0:07:58.389 1G / 19G INFO General (graph_simplification.hpp : 578) All isolated edges shorter than 155 will be removed 0:07:58.389 1G / 19G INFO General (graph_simplification.hpp : 579) Also isolated edges shorter than 0 and coverage smaller than 2 will be removed 0:07:58.480 1G / 19G INFO General (simplification.cpp : 522) Counting average coverage 0:07:58.555 1G / 19G INFO General (simplification.cpp : 527) Average coverage = 17.053 0:07:58.555 1G / 19G INFO StageManager (stage.cpp : 126) STAGE == Contig Output 0:07:58.555 1G / 19G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K21/simplified_contigs.fasta 0:08:00.920 1G / 19G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K21/before_rr.fasta 0:08:03.302 1G / 19G INFO General (contig_output.hpp : 342) Outputting graph to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K21/assembly_graph.fastg 0:08:08.344 1G / 19G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K21/final_contigs.fasta 0:08:10.725 1G / 19G INFO General (launch.hpp : 121) SPAdes finished 0:08:12.250 64M / 19G INFO General (main.cpp : 109) Assembling time: 0 hours 8 minutes 12 seconds == Running assembler: K33 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K33/configs/config.info 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K33/configs/mda_mode.info 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K33/configs/meta_mode.info 0:00:00.000 4M / 4M INFO General (memory_limit.hpp : 47) Memory limit set to 240 Gb 0:00:00.000 4M / 4M INFO General (main.cpp : 88) Starting SPAdes, built from N/A, git revision N/A 0:00:00.000 4M / 4M INFO General (main.cpp : 89) Maximum k-mer length: 128 0:00:00.000 4M / 4M INFO General (main.cpp : 90) Assembling dataset (/home/ec2-user/PGScratch/Varnish/SPAdes_bp8/dataset.info) with K=33 0:00:00.000 4M / 4M INFO General (launch.hpp : 29) SPAdes started 0:00:00.000 4M / 4M INFO General (launch.hpp : 36) Starting from stage: construction 0:00:00.000 4M / 4M INFO General (launch.hpp : 39) Two-step RR enabled: 0 0:00:00.000 4M / 4M INFO StageManager (stage.cpp : 126) STAGE == Construction 0:00:00.000 4M / 4M INFO General (read_converter.hpp : 61) Binary reads detected 0:00:00.000 4M / 4M INFO General (graph_construction.hpp : 120) Constructing DeBruijn graph for k=33 0:00:00.000 4M / 4M INFO General (kmer_splitters.hpp : 130) Splitting kmer instances into 256 buckets. This might take a while. 0:00:00.000 4M / 4M INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:00:00.000 4M / 4M INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.99992 Gb 0:00:00.001 4M / 4M INFO General (kmer_index_builder.hpp : 113) Using cell size of 131072 0:00:01.474 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 1588343 reads 0:00:02.591 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 2881690 reads 0:00:04.241 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 4822290 reads 0:00:05.733 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 6523266 reads 0:00:06.901 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 7843603 reads 0:00:08.931 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 10141660 reads 0:00:10.822 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 12366278 reads 0:00:11.017 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 12511998 reads 0:00:11.017 8G / 9G INFO General (kmer_splitters.hpp : 160) Adding contigs from previous K 0:00:15.867 8G / 9G INFO General (kmer_splitters.hpp : 171) Used 12511998 reads. Maximum read length 155 0:00:15.867 8G / 9G INFO General (kmer_splitters.hpp : 172) Average read length 137.972 0:00:15.867 8G / 9G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:00:35.561 8G / 9G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 94481849 kmers in total. 0:00:35.561 8G / 9G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:00:36.952 8G / 9G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:00:36.952 8G / 9G INFO General (kmer_splitters.hpp : 282) Splitting kmer instances into 256 buckets. This might take a while. 0:00:36.952 8G / 9G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:00:36.952 8G / 9G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.8151 Gb 0:00:36.952 8G / 9G INFO General (kmer_index_builder.hpp : 113) Using cell size of 131072 0:00:44.028 17G / 17G INFO General (kmer_splitters.hpp : 301) Processed 94481849 kmers 0:00:44.028 17G / 17G INFO General (kmer_splitters.hpp : 306) Used 94481849 kmers. 0:00:44.045 17G / 17G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:00:44.444 17G / 17G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 94073999 kmers in total. 0:00:44.444 17G / 17G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:00:45.810 17G / 17G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:00:49.242 17G / 20G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:02:05.625 17G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 30735880 bytes occupied (2.61376 bits per kmer). 0:02:05.726 17G / 20G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 85) Building k-mer extensions from k+1-mers 0:02:13.593 17G / 20G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 89) Building k-mer extensions from k+1-mers finished. 0:02:14.167 156M / 20G INFO Early tip clipping (early_simplification.hpp : 182) Early tip clipping 0:02:27.314 156M / 20G INFO Early tip clipping (early_simplification.hpp : 185) 25465425 34-mers were removed by early tip clipper 0:02:27.314 156M / 20G INFO General (graph_construction.hpp : 136) Condensing graph 0:02:27.314 156M / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 330) Extracting unbranching paths 0:03:10.855 600M / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 344) Extracting unbranching paths finished. 3722788 sequences extracted 0:03:15.143 600M / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 304) Collecting perfect loops 0:03:37.231 600M / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 319) Collecting perfect loops finished. 96 loops collected 0:03:38.494 940M / 20G INFO General (graph_construction.hpp : 141) Building index with from graph 0:03:38.494 940M / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:03:38.494 940M / 20G INFO General (kmer_splitters.hpp : 213) Splitting kmer instances into 16 buckets. This might take a while. 0:03:38.494 940M / 20G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:03:38.494 940M / 20G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 79.694 Gb 0:03:38.494 940M / 20G INFO General (kmer_index_builder.hpp : 113) Using cell size of 2097152 0:03:42.706 1G / 20G INFO General (kmer_splitters.hpp : 224) Processed 1798456 edges 0:03:46.803 1G / 20G INFO General (kmer_splitters.hpp : 224) Processed 3593131 edges 0:03:47.097 1G / 20G INFO General (kmer_splitters.hpp : 224) Processed 3722886 edges 0:03:47.097 1G / 20G INFO General (kmer_splitters.hpp : 229) Used 3722886 sequences. 0:03:47.097 1G / 20G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:03:55.582 1G / 20G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 69016424 kmers in total. 0:03:55.582 1G / 20G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:03:56.522 1G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:04:21.900 1G / 20G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:05:01.023 1G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 22549096 bytes occupied (2.61377 bits per kmer). 0:05:01.780 2G / 20G INFO General (edge_index_builders.hpp : 27) Collecting k-mer coverage information from graph, this takes a while. 0:05:07.213 2G / 20G INFO General (edge_index.hpp : 91) Index refilled 0:05:07.634 2G / 20G INFO General (graph_construction.hpp : 173) Filling coverage index 0:05:07.634 2G / 20G INFO General (edge_index_builders.hpp : 105) Collecting k-mer coverage information from reads, this takes a while. 0:05:34.023 2G / 20G INFO General (graph_construction.hpp : 175) Filling coverage and flanking coverage from index 0:05:49.134 2G / 20G INFO General (construction.cpp : 30) Figured out: read length = 155 0:05:49.134 2G / 20G INFO StageManager (stage.cpp : 126) STAGE == EC Threshold Finding 0:05:51.270 2G / 20G INFO ThresholdFinder (ec_threshold_finder.hpp : 114) Bucket size: 7 0:05:51.270 2G / 20G INFO General (genomic_info_filler.cpp : 104) Average edge coverage: 9.05175 0:05:51.270 2G / 20G INFO General (genomic_info_filler.cpp : 105) Graph threshold: 36 0:05:54.848 2G / 20G INFO General (genomic_info_filler.cpp : 147) EC coverage threshold value was calculated as 9.05175 0:05:54.848 2G / 20G INFO General (genomic_info_filler.cpp : 148) Trusted kmer low bound: 0 0:05:54.848 2G / 20G INFO StageManager (stage.cpp : 126) STAGE == Simplification 0:05:54.916 916M / 20G INFO General (simplification.cpp : 349) Graph simplification started 0:05:54.916 916M / 20G INFO General (simplification.cpp : 98) PROCEDURE == InitialCleaning 0:05:54.916 916M / 20G INFO General (graph_simplification.hpp : 577) Removing isolated edges 0:05:54.916 916M / 20G INFO General (graph_simplification.hpp : 578) All isolated edges shorter than 155 will be removed 0:05:54.916 916M / 20G INFO General (graph_simplification.hpp : 579) Also isolated edges shorter than 0 and coverage smaller than 0 will be removed 0:05:54.916 916M / 20G INFO General (graph_simplification.hpp : 850) Flanking coverage based disconnection disabled 0:05:54.916 916M / 20G INFO General (simplification.cpp : 326) Running Self conjugate edge remover 0:05:55.026 916M / 20G INFO General (simplification.cpp : 326) Running Initial isolated edge remover 0:05:55.505 916M / 20G INFO General (simplification.cpp : 326) Running Initial tip clipper 0:05:57.703 920M / 20G INFO General (simplification.cpp : 326) Running Initial ec remover 0:05:58.047 920M / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:05:58.047 920M / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 1 0:05:58.047 920M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:05:58.734 920M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:06:36.633 928M / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:06:37.352 928M / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 2 0:06:37.352 928M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:06:37.583 928M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:06:37.679 928M / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:06:53.551 936M / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 3 0:06:53.551 936M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:06:53.871 936M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:01.079 936M / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:06.468 936M / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 4 0:07:06.468 936M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:06.536 936M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:09.349 936M / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:09.349 936M / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 5 0:07:09.349 936M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:09.352 936M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:09.355 936M / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:09.355 936M / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 6 0:07:09.355 936M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:09.355 936M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:09.355 936M / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:07:09.355 936M / 20G INFO General (simplification.cpp : 213) PROCEDURE == Post simplification 0:07:09.355 936M / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:07:09.355 936M / 20G INFO General (simplification.cpp : 268) Iteration 0 0:07:09.355 936M / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:14.195 904M / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:14.936 904M / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:14.936 904M / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:15.391 884M / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:16.607 884M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:16.763 880M / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:16.909 880M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:17.720 876M / 20G INFO General (simplification.cpp : 268) Iteration 1 0:07:17.720 876M / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:18.802 876M / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:19.186 876M / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:19.186 876M / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:19.570 876M / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:20.490 876M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:20.499 876M / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:20.500 876M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:20.515 876M / 20G INFO General (simplification.cpp : 268) Iteration 2 0:07:20.515 876M / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:21.457 872M / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:21.818 872M / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:21.818 872M / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:22.190 872M / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:23.066 872M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:23.067 872M / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:23.067 872M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:23.070 872M / 20G INFO General (simplification.cpp : 268) Iteration 3 0:07:23.070 872M / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:23.988 872M / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:24.347 872M / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:24.347 872M / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:24.707 872M / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:25.651 872M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:25.651 872M / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:25.652 872M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:25.652 872M / 20G INFO General (simplification.cpp : 268) Iteration 4 0:07:25.653 872M / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:26.553 872M / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:26.910 872M / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:26.910 872M / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:27.266 872M / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:28.210 872M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:28.210 872M / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:28.210 872M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:28.211 872M / 20G INFO General (simplification.cpp : 268) Iteration 5 0:07:28.211 872M / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:07:29.112 872M / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:07:29.474 872M / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:07:29.474 872M / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:07:29.834 872M / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:07:30.779 872M / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:07:30.779 872M / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:07:30.779 872M / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:07:30.779 872M / 20G INFO General (simplification.cpp : 296) Disrupting self-conjugate edges 0:07:30.961 872M / 20G INFO StageManager (stage.cpp : 126) STAGE == Simplification Cleanup 0:07:30.961 872M / 20G INFO General (graph_simplification.hpp : 577) Removing isolated edges 0:07:30.961 872M / 20G INFO General (graph_simplification.hpp : 578) All isolated edges shorter than 155 will be removed 0:07:30.961 872M / 20G INFO General (graph_simplification.hpp : 579) Also isolated edges shorter than 0 and coverage smaller than 2 will be removed 0:07:31.074 872M / 20G INFO General (simplification.cpp : 522) Counting average coverage 0:07:31.131 872M / 20G INFO General (simplification.cpp : 527) Average coverage = 13.5179 0:07:31.131 872M / 20G INFO StageManager (stage.cpp : 126) STAGE == Contig Output 0:07:31.131 872M / 20G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K33/simplified_contigs.fasta 0:07:33.240 872M / 20G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K33/before_rr.fasta 0:07:35.312 872M / 20G INFO General (contig_output.hpp : 342) Outputting graph to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K33/assembly_graph.fastg 0:07:39.669 872M / 20G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K33/final_contigs.fasta 0:07:41.841 872M / 20G INFO General (launch.hpp : 121) SPAdes finished 0:07:42.976 64M / 20G INFO General (main.cpp : 109) Assembling time: 0 hours 7 minutes 42 seconds == Running assembler: K55 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K55/configs/config.info 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K55/configs/mda_mode.info 0:00:00.000 4M / 4M INFO General (main.cpp : 75) Loading config from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/K55/configs/meta_mode.info 0:00:00.000 4M / 4M INFO General (memory_limit.hpp : 47) Memory limit set to 240 Gb 0:00:00.000 4M / 4M INFO General (main.cpp : 88) Starting SPAdes, built from N/A, git revision N/A 0:00:00.000 4M / 4M INFO General (main.cpp : 89) Maximum k-mer length: 128 0:00:00.000 4M / 4M INFO General (main.cpp : 90) Assembling dataset (/home/ec2-user/PGScratch/Varnish/SPAdes_bp8/dataset.info) with K=55 0:00:00.000 4M / 4M INFO General (launch.hpp : 29) SPAdes started 0:00:00.000 4M / 4M INFO General (launch.hpp : 36) Starting from stage: construction 0:00:00.000 4M / 4M INFO General (launch.hpp : 39) Two-step RR enabled: 1 0:00:00.000 4M / 4M INFO General (launch.hpp : 58) Will need read mapping, kmer mapper will be attached 0:00:00.000 4M / 4M INFO StageManager (stage.cpp : 126) STAGE == Construction 0:00:00.000 4M / 4M INFO General (read_converter.hpp : 61) Binary reads detected 0:00:00.000 4M / 4M INFO General (graph_construction.hpp : 120) Constructing DeBruijn graph for k=55 0:00:00.000 4M / 4M INFO General (kmer_splitters.hpp : 130) Splitting kmer instances into 256 buckets. This might take a while. 0:00:00.000 4M / 4M INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:00:00.001 4M / 4M INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.99992 Gb 0:00:00.001 4M / 4M INFO General (kmer_index_builder.hpp : 113) Using cell size of 131072 0:00:01.766 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 2466263 reads 0:00:03.346 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 4642435 reads 0:00:05.038 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 6999934 reads 0:00:06.942 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 9571382 reads 0:00:09.242 8G / 9G INFO General (kmer_splitters.hpp : 154) Processed 12511998 reads 0:00:09.242 8G / 9G INFO General (kmer_splitters.hpp : 160) Adding contigs from previous K 0:00:14.600 8G / 9G INFO General (kmer_splitters.hpp : 171) Used 12511998 reads. Maximum read length 155 0:00:14.600 8G / 9G INFO General (kmer_splitters.hpp : 172) Average read length 137.972 0:00:14.600 8G / 9G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:00:31.597 8G / 9G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 97260868 kmers in total. 0:00:31.597 8G / 9G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:00:32.959 8G / 9G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:00:32.959 8G / 9G INFO General (kmer_splitters.hpp : 282) Splitting kmer instances into 256 buckets. This might take a while. 0:00:32.959 8G / 9G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:00:32.959 8G / 9G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 4.8151 Gb 0:00:32.959 8G / 9G INFO General (kmer_index_builder.hpp : 113) Using cell size of 131072 0:00:41.639 17G / 17G INFO General (kmer_splitters.hpp : 301) Processed 97260868 kmers 0:00:41.639 17G / 17G INFO General (kmer_splitters.hpp : 306) Used 97260868 kmers. 0:00:41.662 17G / 17G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:00:42.068 17G / 17G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 97326259 kmers in total. 0:00:42.068 17G / 17G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:00:43.428 17G / 17G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:00:47.167 17G / 20G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:02:29.919 17G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 31798448 bytes occupied (2.61376 bits per kmer). 0:02:30.023 17G / 20G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 85) Building k-mer extensions from k+1-mers 0:02:38.365 17G / 20G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 89) Building k-mer extensions from k+1-mers finished. 0:02:38.946 160M / 20G INFO General (graph_construction.hpp : 136) Condensing graph 0:02:38.946 160M / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 330) Extracting unbranching paths 0:03:22.446 1G / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 344) Extracting unbranching paths finished. 11336869 sequences extracted 0:03:28.910 1G / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 304) Collecting perfect loops 0:03:51.940 1G / 20G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 319) Collecting perfect loops finished. 0 loops collected 0:03:56.412 2G / 20G INFO General (graph_construction.hpp : 141) Building index with from graph 0:03:56.412 2G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:03:56.412 2G / 20G INFO General (kmer_splitters.hpp : 213) Splitting kmer instances into 16 buckets. This might take a while. 0:03:56.412 2G / 20G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:03:56.412 2G / 20G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 79.0977 Gb 0:03:56.412 2G / 20G INFO General (kmer_index_builder.hpp : 113) Using cell size of 2097152 0:04:02.041 3G / 20G INFO General (kmer_splitters.hpp : 224) Processed 3800200 edges 0:04:07.487 3G / 20G INFO General (kmer_splitters.hpp : 224) Processed 7600909 edges 0:04:12.795 3G / 20G INFO General (kmer_splitters.hpp : 224) Processed 11336871 edges 0:04:12.795 3G / 20G INFO General (kmer_splitters.hpp : 229) Used 11336871 sequences. 0:04:12.795 3G / 20G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:04:24.708 3G / 20G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 97260868 kmers in total. 0:04:24.708 3G / 20G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:04:26.071 3G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:05:00.480 3G / 20G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:05:59.573 3G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 31777088 bytes occupied (2.61376 bits per kmer). 0:06:00.620 4G / 20G INFO General (edge_index_builders.hpp : 27) Collecting k-mer coverage information from graph, this takes a while. 0:06:09.642 4G / 20G INFO General (edge_index.hpp : 91) Index refilled 0:06:10.066 4G / 20G INFO General (graph_construction.hpp : 173) Filling coverage index 0:06:10.066 4G / 20G INFO General (edge_index_builders.hpp : 105) Collecting k-mer coverage information from reads, this takes a while. 0:06:35.946 4G / 20G INFO General (graph_construction.hpp : 175) Filling coverage and flanking coverage from index 0:07:01.071 4G / 20G INFO General (construction.cpp : 30) Figured out: read length = 155 0:07:01.071 4G / 20G INFO StageManager (stage.cpp : 126) STAGE == EC Threshold Finding 0:07:05.948 4G / 20G INFO ThresholdFinder (ec_threshold_finder.hpp : 114) Bucket size: 6 0:07:05.948 4G / 20G INFO General (genomic_info_filler.cpp : 104) Average edge coverage: 5.33892 0:07:05.948 4G / 20G INFO General (genomic_info_filler.cpp : 105) Graph threshold: 7 0:07:08.251 4G / 20G INFO General (genomic_info_filler.cpp : 147) EC coverage threshold value was calculated as 5.33892 0:07:08.251 4G / 20G INFO General (genomic_info_filler.cpp : 148) Trusted kmer low bound: 0 0:07:08.251 4G / 20G INFO StageManager (stage.cpp : 126) STAGE == Gap Closer 0:07:08.251 4G / 20G INFO General (gap_closer.cpp : 162) Preparing shift maps 0:07:14.817 5G / 20G INFO General (gap_closer.cpp : 122) Processing paired reads (takes a while) 0:07:19.319 5G / 20G INFO General (gap_closer.cpp : 141) Used 3108625 paired reads 0:07:19.319 5G / 20G INFO General (gap_closer.cpp : 143) Merging paired indices 0:07:21.885 4G / 20G INFO GapCloser (gap_closer.cpp : 402) Closing short gaps 0:07:42.403 4G / 20G INFO GapCloser (gap_closer.cpp : 436) Closing short gaps complete: filled 3887 gaps after checking 126298 candidates 0:07:44.917 4G / 20G INFO StageManager (stage.cpp : 126) STAGE == Preliminary Simplification 0:07:45.031 2G / 20G INFO General (simplification.cpp : 349) Graph simplification started 0:07:45.031 2G / 20G INFO General (simplification.cpp : 98) PROCEDURE == InitialCleaning 0:07:45.031 2G / 20G INFO General (graph_simplification.hpp : 577) Removing isolated edges 0:07:45.031 2G / 20G INFO General (graph_simplification.hpp : 578) All isolated edges shorter than 155 will be removed 0:07:45.031 2G / 20G INFO General (graph_simplification.hpp : 579) Also isolated edges shorter than 0 and coverage smaller than 0 will be removed 0:07:45.031 2G / 20G INFO General (simplification.cpp : 326) Running Self conjugate edge remover 0:07:45.465 2G / 20G INFO General (simplification.cpp : 326) Running Initial isolated edge remover 0:07:47.242 2G / 20G INFO General (simplification.cpp : 326) Running Initial tip clipper 0:07:56.195 2G / 20G INFO General (simplification.cpp : 326) Running Initial ec remover 0:07:57.436 2G / 20G INFO General (simplification.cpp : 326) Running Disconnecting edges with low flanking coverage 0:07:58.967 2G / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:07:58.967 2G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 1 0:07:58.967 2G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:08:47.648 2G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:08:54.146 2G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:08:54.489 2G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 2 0:08:54.489 2G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:08:54.536 2G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:08:54.543 2G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:09:01.095 2G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 3 0:09:01.095 2G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:09:01.273 2G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:09:02.863 2G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:09:05.243 2G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 4 0:09:05.243 2G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:09:05.275 2G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:09:05.830 2G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:09:05.830 2G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 5 0:09:05.830 2G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:09:05.831 2G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:09:05.832 2G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:09:05.832 2G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 6 0:09:05.832 2G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:09:05.832 2G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:09:05.832 2G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:09:05.832 2G / 20G INFO General (simplification.cpp : 382) PostSimplification disabled 0:09:05.852 2G / 20G INFO StageManager (stage.cpp : 126) STAGE == Gap Closer 0:09:05.852 2G / 20G INFO General (graph_pack.hpp : 91) Index refill 0:09:05.852 2G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:09:05.852 2G / 20G INFO General (kmer_splitters.hpp : 213) Splitting kmer instances into 16 buckets. This might take a while. 0:09:05.852 2G / 20G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:09:05.852 2G / 20G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 79.2643 Gb 0:09:05.852 2G / 20G INFO General (kmer_index_builder.hpp : 113) Using cell size of 2097152 0:09:09.389 2G / 20G INFO General (kmer_splitters.hpp : 224) Processed 235387 edges 0:09:10.905 2G / 20G INFO General (kmer_splitters.hpp : 224) Processed 339060 edges 0:09:10.905 2G / 20G INFO General (kmer_splitters.hpp : 229) Used 339060 sequences. 0:09:10.905 2G / 20G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:09:16.607 2G / 20G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 47921028 kmers in total. 0:09:16.607 2G / 20G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:09:17.259 2G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:09:31.709 2G / 20G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:09:57.786 2G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 15656824 bytes occupied (2.61377 bits per kmer). 0:09:58.489 3G / 20G INFO General (edge_index_builders.hpp : 27) Collecting k-mer coverage information from graph, this takes a while. 0:10:01.915 3G / 20G INFO General (edge_index.hpp : 91) Index refilled 0:10:01.916 3G / 20G INFO General (gap_closer.cpp : 162) Preparing shift maps 0:10:02.422 3G / 20G INFO General (gap_closer.cpp : 122) Processing paired reads (takes a while) 0:10:08.793 3G / 20G INFO General (gap_closer.cpp : 141) Used 3108625 paired reads 0:10:08.793 3G / 20G INFO General (gap_closer.cpp : 143) Merging paired indices 0:10:09.608 3G / 20G INFO GapCloser (gap_closer.cpp : 402) Closing short gaps 0:10:10.547 3G / 20G INFO GapCloser (gap_closer.cpp : 436) Closing short gaps complete: filled 581 gaps after checking 47500 candidates 0:10:11.618 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Simplification Cleanup 0:10:11.618 3G / 20G INFO General (graph_simplification.hpp : 577) Removing isolated edges 0:10:11.618 3G / 20G INFO General (graph_simplification.hpp : 578) All isolated edges shorter than 155 will be removed 0:10:11.618 3G / 20G INFO General (graph_simplification.hpp : 579) Also isolated edges shorter than 0 and coverage smaller than 2 will be removed 0:10:17.505 3G / 20G INFO General (simplification.cpp : 522) Counting average coverage 0:10:17.584 3G / 20G INFO General (simplification.cpp : 527) Average coverage = 10.4394 0:10:17.584 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Preliminary Paired Information Counting 0:10:19.192 3G / 20G INFO General (graph_pack.hpp : 100) Normalizing k-mer map. Total 1562476 kmers to process 0:10:20.415 3G / 20G INFO General (graph_pack.hpp : 102) Normalizing done 0:10:20.415 3G / 20G INFO General (pair_info_count.cpp : 173) Min edge length for estimation: 1000 0:10:20.416 3G / 20G INFO General (pair_info_count.cpp : 187) Estimating insert size for library #0 0:10:20.416 3G / 20G INFO General (pair_info_count.cpp : 23) Estimating insert size (takes a while) 0:10:20.416 3G / 20G INFO General (short_read_mapper.hpp : 93) Selecting usual mapper 0:10:26.907 3G / 20G INFO General (sequence_mapper_notifier.h: 94) Total 3108625 reads processed 0:10:26.909 3G / 20G INFO General (pair_info_count.cpp : 36) 1419477 paired reads (45.6625% of all) aligned to long edges 0:10:26.910 3G / 20G INFO General (pair_info_count.cpp : 209) Insert size = 244.65, deviation = 137.462, left quantile = 93, right quantile = 433, read length = 155 0:10:26.910 3G / 20G INFO General (pair_info_count.cpp : 231) Mapping library #0 0:10:26.910 3G / 20G INFO General (pair_info_count.cpp : 243) Mapping paired reads (takes a while) 0:10:26.910 3G / 20G INFO General (pair_info_count.cpp : 87) Left insert size qauntile 93, right insert size quantile 433 0:10:26.910 3G / 20G INFO General (short_read_mapper.hpp : 93) Selecting usual mapper 0:10:39.838 5G / 20G INFO General (sequence_mapper_notifier.h: 94) Total 3108625 reads processed 0:11:32.184 5G / 20G INFO General (split_graph_pair_info.hpp : 356) Threshold for paired information 0.0161651 0:11:53.480 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Preliminary Distance Estimation 0:11:53.480 3G / 20G INFO General (distance_estimation.cpp : 226) Processing library #0 0:11:53.480 3G / 20G INFO General (distance_estimation.cpp : 132) Weight Filter Done 0:11:53.480 3G / 20G INFO DistanceEstimator (distance_estimation.hpp : 185) Using SIMPLE distance estimator 0:12:02.247 3G / 20G INFO General (distance_estimation.cpp : 35) Filtering info 0:12:02.247 3G / 20G INFO General (pair_info_filters.hpp : 242) Start filtering; index size: 544809 0:12:02.601 3G / 20G INFO General (pair_info_filters.hpp : 263) Done filtering 0:12:02.602 3G / 20G INFO General (distance_estimation.cpp : 174) Refining clustered pair information 0:12:02.723 3G / 20G INFO General (distance_estimation.cpp : 176) The refining of clustered pair information has been finished 0:12:02.723 3G / 20G INFO General (distance_estimation.cpp : 178) Improving paired information 0:12:03.341 3G / 20G INFO PairInfoImprover (pair_info_improver.hpp : 103) Paired info stats: missing = 11874; contradictional = 0 0:12:03.950 3G / 20G INFO PairInfoImprover (pair_info_improver.hpp : 103) Paired info stats: missing = 1056; contradictional = 0 0:12:03.950 3G / 20G INFO General (distance_estimation.cpp : 185) Filling scaffolding index 0:12:03.950 3G / 20G INFO DistanceEstimator (distance_estimation.hpp : 185) Using SMOOTHING distance estimator 0:12:13.758 3G / 20G INFO General (distance_estimation.cpp : 35) Filtering info 0:12:13.758 3G / 20G INFO General (pair_info_filters.hpp : 242) Start filtering; index size: 377336 0:12:13.953 3G / 20G INFO General (pair_info_filters.hpp : 263) Done filtering 0:12:13.954 3G / 20G INFO General (distance_estimation.cpp : 230) Clearing raw paired index 0:12:15.028 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Preliminary Repeat Resolving 0:12:15.028 3G / 20G INFO General (repeat_resolving.cpp : 61) Setting up preliminary path extend settings 0:12:15.028 3G / 20G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/before_rr.fasta 0:12:17.197 3G / 20G INFO General (contig_output.hpp : 342) Outputting graph to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/assembly_graph.fastg 0:12:21.658 3G / 20G INFO General (repeat_resolving.cpp : 78) Using Path-Extend repeat resolving 0:12:21.658 3G / 20G INFO General (path_extend_launch.hpp :1056) ExSPAnder repeat resolving tool started 0:12:22.357 3G / 20G INFO General (path_extend_launch.hpp :1094) SUBSTAGE = paired-end libraries 0:12:22.459 3G / 20G INFO General (path_extend_launch.hpp : 874) Using 1 paired-end library 0:12:22.459 3G / 20G INFO General (path_extend_launch.hpp : 875) Using 1 paired-end scaffolding library 0:12:22.459 3G / 20G INFO General (path_extend_launch.hpp : 876) Using 0 mate-pair libraries 0:12:22.459 3G / 20G INFO General (path_extend_launch.hpp : 877) Using 0 single read libraries 0:12:22.459 3G / 20G INFO General (path_extend_launch.hpp : 878) Scaffolder is on 0:12:24.290 3G / 20G INFO General (path_extend_launch.hpp :1115) Growing paths using paired-end and long single reads 0:12:24.290 3G / 20G INFO General (path_extend_launch.hpp :1116) Multi path extend is off 0:12:24.290 3G / 20G INFO General (path_extend_launch.hpp :1117) Overlap removal is off 0:12:24.290 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 0 paths from 120990 (0%) 0:12:24.337 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 128 paths from 120990 (0%) 0:12:24.380 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 256 paths from 120990 (0%) 0:12:24.477 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 512 paths from 120990 (0%) 0:12:24.659 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 1024 paths from 120990 (0%) 0:12:25.082 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 2048 paths from 120990 (1%) 0:12:25.783 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 4096 paths from 120990 (3%) 0:12:26.792 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 8192 paths from 120990 (6%) 0:12:27.345 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 12100 paths from 120990 (10%) 0:12:27.757 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 16384 paths from 120990 (13%) 0:12:28.270 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 24200 paths from 120990 (20%) 0:12:28.626 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 32768 paths from 120990 (27%) 0:12:28.749 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 36300 paths from 120990 (30%) 0:12:29.197 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 48400 paths from 120990 (40%) 0:12:29.546 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 60500 paths from 120990 (50%) 0:12:29.718 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 65536 paths from 120990 (54%) 0:12:29.900 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 72600 paths from 120990 (60%) 0:12:30.201 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 84700 paths from 120990 (70%) 0:12:30.922 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 96800 paths from 120990 (80%) 0:12:35.597 4G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 108900 paths from 120990 (90%) 0:12:37.466 4G / 20G INFO General (path_extend_launch.hpp : 874) Using 1 paired-end library 0:12:37.466 4G / 20G INFO General (path_extend_launch.hpp : 875) Using 1 paired-end scaffolding library 0:12:37.466 4G / 20G INFO General (path_extend_launch.hpp : 876) Using 0 mate-pair libraries 0:12:37.466 4G / 20G INFO General (path_extend_launch.hpp : 877) Using 0 single read libraries 0:12:37.466 4G / 20G INFO General (path_extend_launch.hpp : 878) Scaffolder is on 0:12:39.485 4G / 20G INFO PathExtendIO (pe_io.hpp : 214) Writing contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/final_contigs 0:12:44.842 4G / 20G INFO General (path_extend_launch.hpp : 263) Traversing tandem repeats 0:12:48.050 4G / 20G INFO PathExtendIO (pe_io.hpp : 214) Writing contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/scaffolds 0:12:53.569 3G / 20G INFO General (repeat_resolving.cpp : 85) Restoring initial path extend settings 0:12:53.569 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Second Phase Setup 0:12:53.569 3G / 20G INFO General (second_phase_setup.cpp : 20) Preparing second phase 0:12:53.935 3G / 20G INFO General (second_phase_setup.cpp : 27) Moving preliminary contigs from /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/final_contigs.fasta to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/first_pe_contigs.fasta 0:12:53.936 3G / 20G INFO General (second_phase_setup.cpp : 39) Ready to run second phase 0:12:53.936 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Simplification 0:12:53.990 1G / 20G INFO General (simplification.cpp : 349) Graph simplification started 0:12:53.990 1G / 20G INFO General (simplification.cpp : 98) PROCEDURE == InitialCleaning 0:12:53.990 1G / 20G INFO General (simplification.cpp : 71) Most init cleaning disabled on main iteration 0:12:53.990 1G / 20G INFO General (simplification.cpp : 326) Running Self conjugate edge remover 0:12:54.041 1G / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:12:54.041 1G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 1 0:12:54.041 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:12:54.134 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:12:54.940 1G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:12:54.992 1G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 2 0:12:54.992 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:12:54.993 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:12:54.993 1G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:12:54.994 1G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 3 0:12:54.994 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:12:54.994 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:12:54.996 1G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:12:54.996 1G / 20G INFO General (simplification.cpp : 372) PROCEDURE == Simplification cycle, iteration 4 0:12:54.996 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:12:54.996 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:12:54.998 1G / 20G INFO General (simplification.cpp : 326) Running Low coverage edge remover 0:12:54.998 1G / 20G INFO General (simplification.cpp : 213) PROCEDURE == Post simplification 0:12:54.998 1G / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:12:54.998 1G / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:12:54.998 1G / 20G INFO General (graph_simplification.hpp : 763) Creating parallel br instance 0:12:54.998 1G / 20G INFO General (simplification.cpp : 268) Iteration 0 0:12:54.998 1G / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:12:56.192 1G / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:12:56.595 1G / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:12:56.595 1G / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:12:57.007 1G / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:12:58.025 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:12:58.104 1G / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:12:58.179 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:12:58.243 1G / 20G INFO General (simplification.cpp : 326) Running Final bulge remover 0:12:58.368 1G / 20G INFO General (simplification.cpp : 326) Running Yet another final bulge remover 0:12:58.464 1G / 20G INFO General (simplification.cpp : 268) Iteration 1 0:12:58.464 1G / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:12:58.848 1G / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:12:59.142 1G / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:12:59.142 1G / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:12:59.555 1G / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:13:00.561 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:13:00.562 1G / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:13:00.563 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:13:00.565 1G / 20G INFO General (simplification.cpp : 326) Running Final bulge remover 0:13:00.566 1G / 20G INFO General (simplification.cpp : 326) Running Yet another final bulge remover 0:13:00.568 1G / 20G INFO General (simplification.cpp : 268) Iteration 2 0:13:00.568 1G / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:13:00.958 1G / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:13:01.254 1G / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:13:01.254 1G / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:13:01.642 1G / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:13:02.603 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:13:02.603 1G / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:13:02.603 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:13:02.605 1G / 20G INFO General (simplification.cpp : 326) Running Final bulge remover 0:13:02.605 1G / 20G INFO General (simplification.cpp : 326) Running Yet another final bulge remover 0:13:02.606 1G / 20G INFO General (simplification.cpp : 268) Iteration 3 0:13:02.606 1G / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:13:02.986 1G / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:13:03.274 1G / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:13:03.274 1G / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:13:03.651 1G / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:13:04.618 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:13:04.618 1G / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:13:04.618 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:13:04.619 1G / 20G INFO General (simplification.cpp : 326) Running Final bulge remover 0:13:04.619 1G / 20G INFO General (simplification.cpp : 326) Running Yet another final bulge remover 0:13:04.619 1G / 20G INFO General (simplification.cpp : 268) Iteration 4 0:13:04.619 1G / 20G INFO General (graph_simplification.hpp : 441) Removing relatively low covered connections 0:13:05.010 1G / 20G INFO General (graph_simplification.hpp : 473) Disconnecting edges with relatively low coverage 0:13:05.307 1G / 20G INFO General (graph_simplification.hpp : 546) Complex tip clipping 0:13:05.307 1G / 20G INFO ComplexTipClipper (complex_tip_clipper.hpp : 97) Complex tip clipper started 0:13:05.692 1G / 20G INFO General (graph_simplification.hpp : 492) Removing complex bulges 0:13:06.654 1G / 20G INFO General (simplification.cpp : 326) Running Tip clipper 0:13:06.654 1G / 20G INFO General (simplification.cpp : 326) Running Final tip clipper 0:13:06.654 1G / 20G INFO General (simplification.cpp : 326) Running Bulge remover 0:13:06.654 1G / 20G INFO General (simplification.cpp : 326) Running Final bulge remover 0:13:06.654 1G / 20G INFO General (simplification.cpp : 326) Running Yet another final bulge remover 0:13:06.654 1G / 20G INFO General (simplification.cpp : 296) Disrupting self-conjugate edges 0:13:06.810 1G / 20G INFO StageManager (stage.cpp : 126) STAGE == Paired Information Counting 0:13:08.032 1G / 20G INFO General (graph_pack.hpp : 91) Index refill 0:13:08.033 1G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 346) Building kmer index 0:13:08.033 1G / 20G INFO General (kmer_splitters.hpp : 213) Splitting kmer instances into 16 buckets. This might take a while. 0:13:08.033 1G / 20G INFO General (file_limit.hpp : 30) Open file limit set to 1024 0:13:08.033 1G / 20G INFO General (kmer_index_builder.hpp : 105) Memory available for splitting buffers: 79.3906 Gb 0:13:08.033 1G / 20G INFO General (kmer_index_builder.hpp : 113) Using cell size of 2097152 0:13:11.507 2G / 20G INFO General (kmer_splitters.hpp : 224) Processed 149606 edges 0:13:12.302 2G / 20G INFO General (kmer_splitters.hpp : 224) Processed 185165 edges 0:13:12.302 2G / 20G INFO General (kmer_splitters.hpp : 229) Used 185165 sequences. 0:13:12.302 2G / 20G INFO General (kmer_index_builder.hpp : 232) Starting k-mer counting. 0:13:17.124 2G / 20G INFO General (kmer_index_builder.hpp : 238) K-mer counting done. There are 41555649 kmers in total. 0:13:17.124 2G / 20G INFO General (kmer_index_builder.hpp : 240) Merging temporary buckets. 0:13:17.676 2G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 355) Building perfect hash indices 0:13:29.603 2G / 20G INFO General (kmer_index_builder.hpp : 256) Merging final buckets. 0:13:52.104 2G / 20G INFO K-mer Index Building (kmer_index_builder.hpp : 401) Index built. Total 13577144 bytes occupied (2.61378 bits per kmer). 0:13:52.574 2G / 20G INFO General (edge_index_builders.hpp : 27) Collecting k-mer coverage information from graph, this takes a while. 0:13:55.503 2G / 20G INFO General (edge_index.hpp : 91) Index refilled 0:13:55.503 2G / 20G INFO General (graph_pack.hpp : 100) Normalizing k-mer map. Total 1710706 kmers to process 0:13:56.840 2G / 20G INFO General (graph_pack.hpp : 102) Normalizing done 0:13:56.840 2G / 20G INFO General (pair_info_count.cpp : 173) Min edge length for estimation: 1000 0:13:56.840 2G / 20G INFO General (pair_info_count.cpp : 187) Estimating insert size for library #0 0:13:56.840 2G / 20G INFO General (pair_info_count.cpp : 23) Estimating insert size (takes a while) 0:13:56.840 2G / 20G INFO General (short_read_mapper.hpp : 93) Selecting usual mapper 0:14:03.767 2G / 20G INFO General (sequence_mapper_notifier.h: 94) Total 3108625 reads processed 0:14:03.770 2G / 20G INFO General (pair_info_count.cpp : 36) 1946477 paired reads (62.6154% of all) aligned to long edges 0:14:03.771 2G / 20G INFO General (pair_info_count.cpp : 209) Insert size = 252.132, deviation = 141.525, left quantile = 93, right quantile = 433, read length = 155 0:14:03.771 2G / 20G INFO General (pair_info_count.cpp : 231) Mapping library #0 0:14:03.771 2G / 20G INFO General (pair_info_count.cpp : 243) Mapping paired reads (takes a while) 0:14:03.771 2G / 20G INFO General (pair_info_count.cpp : 87) Left insert size qauntile 93, right insert size quantile 433 0:14:03.771 2G / 20G INFO General (short_read_mapper.hpp : 93) Selecting usual mapper 0:14:13.508 4G / 20G INFO General (sequence_mapper_notifier.h: 94) Total 3108625 reads processed 0:14:50.651 4G / 20G INFO General (split_graph_pair_info.hpp : 356) Threshold for paired information 0.018863 0:15:07.063 3G / 20G INFO General (pair_info_count.cpp : 225) Mapping contigs library #1 0:15:07.063 3G / 20G INFO General (short_read_mapper.hpp : 93) Selecting usual mapper 0:15:11.028 3G / 20G INFO General (sequence_mapper_notifier.h: 94) Total 104420 reads processed 0:15:11.117 3G / 20G INFO StageManager (stage.cpp : 126) STAGE == Distance Estimation 0:15:11.117 3G / 20G INFO General (distance_estimation.cpp : 226) Processing library #0 0:15:11.117 3G / 20G INFO General (distance_estimation.cpp : 132) Weight Filter Done 0:15:11.117 3G / 20G INFO DistanceEstimator (distance_estimation.hpp : 185) Using SIMPLE distance estimator 0:15:11.598 3G / 20G INFO General (distance_estimation.cpp : 35) Filtering info 0:15:11.598 3G / 20G INFO General (pair_info_filters.hpp : 242) Start filtering; index size: 284777 0:15:11.808 3G / 20G INFO General (pair_info_filters.hpp : 263) Done filtering 0:15:11.808 3G / 20G INFO General (distance_estimation.cpp : 174) Refining clustered pair information 0:15:11.881 3G / 20G INFO General (distance_estimation.cpp : 176) The refining of clustered pair information has been finished 0:15:11.881 3G / 20G INFO General (distance_estimation.cpp : 178) Improving paired information 0:15:12.224 3G / 20G INFO PairInfoImprover (pair_info_improver.hpp : 103) Paired info stats: missing = 3687; contradictional = 0 0:15:12.551 3G / 20G INFO PairInfoImprover (pair_info_improver.hpp : 103) Paired info stats: missing = 198; contradictional = 0 0:15:12.551 3G / 20G INFO General (distance_estimation.cpp : 185) Filling scaffolding index 0:15:12.551 3G / 20G INFO DistanceEstimator (distance_estimation.hpp : 185) Using SMOOTHING distance estimator 0:15:18.332 3G / 20G INFO General (distance_estimation.cpp : 35) Filtering info 0:15:18.332 3G / 20G INFO General (pair_info_filters.hpp : 242) Start filtering; index size: 246441 0:15:18.458 3G / 20G INFO General (pair_info_filters.hpp : 263) Done filtering 0:15:18.459 3G / 20G INFO General (distance_estimation.cpp : 230) Clearing raw paired index 0:15:18.805 2G / 20G INFO StageManager (stage.cpp : 126) STAGE == Repeat Resolving 0:15:18.805 2G / 20G INFO General (contig_output.hpp : 309) Outputting contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/before_rr.fasta 0:15:21.797 2G / 20G INFO General (contig_output.hpp : 342) Outputting graph to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/assembly_graph.fastg 0:15:28.235 2G / 20G INFO General (repeat_resolving.cpp : 78) Using Path-Extend repeat resolving 0:15:28.235 2G / 20G INFO General (path_extend_launch.hpp :1056) ExSPAnder repeat resolving tool started 0:15:28.780 2G / 20G INFO General (path_extend_launch.hpp :1094) SUBSTAGE = paired-end libraries 0:15:28.870 2G / 20G INFO General (path_extend_launch.hpp : 381) resolvable_repeat_length_bound set to 10000 0:15:29.312 2G / 20G INFO General (path_extend_launch.hpp : 874) Using 1 paired-end library 0:15:29.312 2G / 20G INFO General (path_extend_launch.hpp : 875) Using 1 paired-end scaffolding library 0:15:29.312 2G / 20G INFO General (path_extend_launch.hpp : 876) Using 0 mate-pair libraries 0:15:29.312 2G / 20G INFO General (path_extend_launch.hpp : 877) Using 1 single read library 0:15:29.312 2G / 20G INFO General (path_extend_launch.hpp : 878) Scaffolder is on 0:15:29.312 2G / 20G INFO General (path_extend_launch.hpp : 881) Using additional coordinated coverage extender 0:15:30.644 3G / 20G INFO General (path_extend_launch.hpp :1115) Growing paths using paired-end and long single reads 0:15:30.644 3G / 20G INFO General (path_extend_launch.hpp :1116) Multi path extend is off 0:15:30.644 3G / 20G INFO General (path_extend_launch.hpp :1117) Overlap removal is on 0:15:30.644 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 0 paths from 92245 (0%) 0:15:30.735 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 128 paths from 92245 (0%) 0:15:30.815 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 256 paths from 92245 (0%) 0:15:30.932 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 512 paths from 92245 (0%) 0:15:31.220 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 1024 paths from 92245 (1%) 0:15:31.694 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 2048 paths from 92245 (2%) 0:15:32.317 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 4096 paths from 92245 (4%) 0:15:33.009 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 8192 paths from 92245 (8%) 0:15:33.097 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 9225 paths from 92245 (10%) 0:15:33.495 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 16384 paths from 92245 (17%) 0:15:33.586 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 18450 paths from 92245 (20%) 0:15:33.861 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 27675 paths from 92245 (30%) 0:15:33.985 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 32768 paths from 92245 (35%) 0:15:34.086 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 36900 paths from 92245 (40%) 0:15:34.288 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 46125 paths from 92245 (50%) 0:15:34.477 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 55350 paths from 92245 (60%) 0:15:34.663 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 64575 paths from 92245 (70%) 0:15:34.681 3G / 20G INFO PathExtender (path_extender.hpp : 950) Processed 65536 paths from 92245 (71%) 0:15:34.847 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 73800 paths from 92245 (80%) 0:15:35.261 3G / 20G INFO PathExtender (path_extender.hpp : 952) Processed 83025 paths from 92245 (90%) 0:15:39.180 3G / 20G INFO General (path_extend_launch.hpp : 381) resolvable_repeat_length_bound set to 10000 0:15:39.619 3G / 20G INFO General (path_extend_launch.hpp : 874) Using 1 paired-end library 0:15:39.619 3G / 20G INFO General (path_extend_launch.hpp : 875) Using 1 paired-end scaffolding library 0:15:39.619 3G / 20G INFO General (path_extend_launch.hpp : 876) Using 0 mate-pair libraries 0:15:39.619 3G / 20G INFO General (path_extend_launch.hpp : 877) Using 1 single read library 0:15:39.619 3G / 20G INFO General (path_extend_launch.hpp : 878) Scaffolder is on 0:15:39.619 3G / 20G INFO General (path_extend_launch.hpp : 881) Using additional coordinated coverage extender 0:15:42.646 3G / 20G INFO PathExtendIO (pe_io.hpp : 214) Writing contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/final_contigs 0:15:46.061 3G / 20G INFO General (path_extend_launch.hpp : 263) Traversing tandem repeats 0:15:49.289 3G / 20G INFO PathExtendIO (pe_io.hpp : 214) Writing contigs to /home/ec2-user/PGScratch/Varnish/SPAdes_bp8//K55/scaffolds 0:15:54.263 2G / 20G INFO General (launch.hpp : 121) SPAdes finished 0:15:57.844 172M / 20G INFO General (main.cpp : 109) Assembling time: 0 hours 15 minutes 57 seconds ===== Assembling finished. Used k-mer sizes: 21, 33, 55 * Corrected reads are in /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/corrected/ * Assembled contigs are in /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/contigs.fasta * Assembled scaffolds are in /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/scaffolds.fasta * Assembly graph is in /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/assembly_graph.fastg * Paths in the assembly graph corresponding to the contigs are in /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/contigs.paths * Paths in the assembly graph corresponding to the scaffolds are in /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/scaffolds.paths ======= SPAdes pipeline finished. SPAdes log can be found here: /home/ec2-user/PGScratch/Varnish/SPAdes_bp8/spades.log Thank you for using SPAdes!