1 Environment 1.1 System 1.1.1 System Platform Platform: x86-64 Version: 12.04 LTS OS: Ubuntu (64-bit) 1.1.2 Hardware CPU(s): 16 Threads: 32 RAM: 128G Hard disk: <2T 1.2 Qiime 1.2.1 Qiime version Version: 1.8.0 1.2.2 System information Platform: linux2 Python/GCC version: 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] Python executable: /home/qiime/miniconda3/bin/python 1.2.3 QIIME default reference information QIIME library version: 1.9.1 QIIME script version: 1.9.1 qiime-default-reference version: 0.1.3 NumPy version: 1.11.0 SciPy version: 0.17.1 pandas version: 0.17.1 matplotlib version: 1.4.3 biom-format version: 2.1.5 qcli version: 0.1.1 pyqi version: 0.3.2 scikit-bio version: 0.2.3 PyNAST version: 1.2.2 Emperor version: 0.9.51 burrito version: 0.9.1 burrito-fillings version: 0.1.1 sortmerna version: SortMeRNA version 2.0, 29/11/2014 sumaclust version: SUMACLUST Version 1.0.00 swarm version: Swarm 1.2.19 [Dec 5 2015 16:48:11] gdata: Installed. 1.2.4 QIIME config values (print_qiime_config.py -tf) For definitions of these settings and to learn how to configure QIIME, see here: http://qiime.org/install/qiime_config.html http://qiime.org/tutorials/parallel_qiime.html QIIME config values For definitions of these settings and to learn how to configure QIIME, see here: http://qiime.org/install/qiime_config.html http://qiime.org/tutorials/parallel_qiime.html blastmat_dir: None pick_otus_reference_seqs_fp: /usr/local/lib/python2.7/sitepackages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta python_exe_fp: python sc_queue: all.q topiaryexplorer_project_dir: None pynast_template_alignment_fp: /usr/local/data/core_set_aligned.fasta.imputed cluster_jobs_fp: None pynast_template_alignment_blastdb: None assign_taxonomy_reference_seqs_fp: /usr/local/lib/python2.7/sitepackages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta torque_queue: friendlyq qiime_test_data_dir: None template_alignment_lanemask_fp: /usr/local/data/lanemask_in_1s_and_0s.txt jobs_to_start: 1 slurm_time: None cloud_environment: False qiime_scripts_dir: /usr/local/bin denoiser_min_per_core: 50 working_dir: None assign_taxonomy_id_to_taxonomy_fp: /usr/local/lib/python2.7/sitepackages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt temp_dir: /tmp/ slurm_memory: None slurm_queue: None blastall_fp: blastall seconds_to_sleep: 2 1.3 PICRUSt 1.3.1 PICRUSt version Version: 1.1.3 1.3.2 System information (print_picrust_config.py) System information ================== Platform: linux2 Python/GCC version: 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] Python executable: /home/qiime/miniconda3/bin/python Dependency versions =================== NumPy version: 1.11.3 biom-format version: 2.1.7 PyCogent version: 1.9 PICRUSt version: 1.1.3 PICRUSt script version: 1.1.3 2. Data 2.1 original sequences format of sequences: sff Number of sff files: 7 sff_filenames: FISH1.sff (ENA accession no. ERR3153880) : FISH2.sff (ERR3153881) : FISH3.sff (ERR3153882) : FISH4.sff (ERR3153883) : FISH5.sff (ERR3153884) : FISH6.sff (ERR3153885) : FISH7.sff (ERR3153886) 3. Scripts 3.1 qiime scripts $ process_sff.py -i FISH1.sff -f $ process_sff.py -i FISH2.sff -f $ process_sff.py -i FISH3.sff -f $ process_sff.py -i FISH4.sff -f $ process_sff.py -i FISH5.sff -f $ process_sff.py -i FISH6.sff -f $ process_sff.py -i FISH7.sff -f $ validate_mapping_file.py -m FISH1_mapping.txt -o mapping_output/FISH1/ -v $ validate_mapping_file.py -m FISH2_mapping.txt -o mapping_output/FISH2/ -v $ validate_mapping_file.py -m FISH3_mapping.txt -o mapping_output/FISH3/ -v $ validate_mapping_file.py -m FISH4_mapping.txt -o mapping_output/FISH4/ -v $ validate_mapping_file.py -m FISH5_mapping.txt -o mapping_output/FISH5/ -v $ validate_mapping_file.py -m FISH6_mapping.txt -o mapping_output/FISH6/ -v $ validate_mapping_file.py -m FISH7_mapping.txt -o mapping_output/FISH7/ -v $ split_libraries.py -m FISH1_mapping.txt -f FISH1.fna -q FISH1.qual -o split_output/FISH1/ -b 10 -M 2 -n 1000000 $ split_libraries.py -m FISH2_mapping.txt -f FISH2.fna -q FISH2.qual -o split_output/FISH2/ -b 10 -M 2 -n 2000000 $ split_libraries.py -m FISH3_mapping.txt -f FISH3.fna -q FISH3.qual -o split_output/FISH3/ -b 10 -M 2 -n 3000000 $ split_libraries.py -m FISH4_mapping.txt -f FISH4.fna -q FISH4.qual -o split_output/FISH4/ -b 10 -M 2 -n 4000000 $ split_libraries.py -m FISH5_mapping.txt -f FISH5.fna -q FISH5.qual -o split_output/FISH5/ -b 10 -M 2 -n 5000000 $ split_libraries.py -m FISH6_mapping.txt -f FISH6.fna -q FISH6.qual -o split_output/FISH6/ -b 10 -M 2 -n 6000000 $ split_libraries.py -m FISH7_mapping.txt -f FISH7.fna -q FISH7.qual -o split_output/FISH7/ -b 10 -M 2 -n 7000000 $ denoise_wrapper.py -i FISH1.sff.txt -f split_output/FISH1/seqs.fna -o denoised/FISH1/ -m FISH1_mapping.txt -n 32 --titanium $ denoise_wrapper.py -i FISH2.sff.txt -f split_output/FISH2/seqs.fna -o denoised/FISH2/ -m FISH2_mapping.txt -n 32 --titanium $ denoise_wrapper.py -i FISH3.sff.txt -f split_output/FISH3/seqs.fna -o denoised/FISH3/ -m FISH3_mapping.txt -n 32 --titanium $ denoise_wrapper.py -i FISH4.sff.txt -f split_output/FISH4/seqs.fna -o denoised/FISH4/ -m FISH4_mapping.txt -n 32 --titanium $ denoise_wrapper.py -i FISH5.sff.txt -f split_output/FISH5/seqs.fna -o denoised/FISH5/ -m FISH5_mapping.txt -n 32 --titanium $ denoise_wrapper.py -i FISH6.sff.txt -f split_output/FISH6/seqs.fna -o denoised/FISH6/ -m FISH6_mapping.txt -n 32 --titanium $ denoise_wrapper.py -i FISH7.sff.txt -f split_output/FISH7/seqs.fna -o denoised/FISH7/ -m FISH7_mapping.txt -n 32 --titanium $ inflate_denoiser_output.py -c denoised/FISH1/centroids.fasta -s denoised/FISH1/singletons.fasta -f split_output/FISH1/seqs.fna -d denoised/FISH1/denoiser_mapping.txt -o FISH1_inflated_seqs.fasta $ inflate_denoiser_output.py -c denoised/FISH2/centroids.fasta -s denoised/FISH2/singletons.fasta -f split_output/FISH2/seqs.fna -d denoised/FISH2/denoiser_mapping.txt -o FISH2_inflated_seqs.fasta $ inflate_denoiser_output.py -c denoised/FISH3/centroids.fasta -s denoised/FISH3/singletons.fasta -f split_output/FISH3/seqs.fna -d denoised/FISH3/denoiser_mapping.txt -o FISH3_inflated_seqs.fasta $ inflate_denoiser_output.py -c denoised/FISH4/centroids.fasta -s denoised/FISH4/singletons.fasta -f split_output/FISH4/seqs.fna -d denoised/FISH4/denoiser_mapping.txt -o FISH4_inflated_seqs.fasta $ inflate_denoiser_output.py -c denoised/FISH5/centroids.fasta -s denoised/FISH5/singletons.fasta -f split_output/FISH5/seqs.fna -d denoised/FISH5/denoiser_mapping.txt -o FISH5_inflated_seqs.fasta $ inflate_denoiser_output.py -c denoised/FISH6/centroids.fasta -s denoised/FISH6/singletons.fasta -f split_output/FISH6/seqs.fna -d denoised/FISH6/denoiser_mapping.txt -o FISH6_inflated_seqs.fasta $ inflate_denoiser_output.py -c denoised/FISH7/centroids.fasta -s denoised/FISH7/singletons.fasta -f split_output/FISH7/seqs.fna -d denoised/FISH7/denoiser_mapping.txt -o FISH7_inflated_seqs.fasta $ truncate_reverse_primer.py -f FISH1_inflated_seqs.fasta -m FISH1_mapping.txt -o truncate_rv/FISH1/ -M 1 $ truncate_reverse_primer.py -f FISH2_inflated_seqs.fasta -m FISH2_mapping.txt -o truncate_rv/FISH2/ -M 1 $ truncate_reverse_primer.py -f FISH3_inflated_seqs.fasta -m FISH3_mapping.txt -o truncate_rv/FISH3/ -M 1 $ truncate_reverse_primer.py -f FISH4_inflated_seqs.fasta -m FISH4_mapping.txt -o truncate_rv/FISH4/ -M 1 $ truncate_reverse_primer.py -f FISH5_inflated_seqs.fasta -m FISH5_mapping.txt -o truncate_rv/FISH5/ -M 1 $ truncate_reverse_primer.py -f FISH6_inflated_seqs.fasta -m FISH6_mapping.txt -o truncate_rv/FISH6/ -M 1 $ truncate_reverse_primer.py -f FISH7_inflated_seqs.fasta -m FISH7_mapping.txt -o truncate_rv/FISH7/ -M 1 $ $ chmod +x usearch7.0/usearch7.0.1090_i86linux32 (Downloaded from ) $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH1/truncated_rv_FISH1.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH1/OUTPUT.uchime -nonchimeras usearch7.0/FISH1/nonchimeras_FISH1.fna -strand plus -threads 32 $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH2/truncated_rv_FISH2.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH2/OUTPUT.uchime -nonchimeras usearch7.0/FISH2/nonchimeras_FISH2.fna -strand plus -threads 32 $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH3/truncated_rv_FISH3.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH3/OUTPUT.uchime -nonchimeras usearch7.0/FISH3/nonchimeras_FISH3.fna -strand plus -threads 32 $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH4/truncated_rv_FISH4.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH4/OUTPUT.uchime -nonchimeras usearch7.0/FISH4/nonchimeras_FISH4.fna -strand plus -threads 32 $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH5/truncated_rv_FISH5.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH5/OUTPUT.uchime -nonchimeras usearch7.0/FISH5/nonchimeras_FISH5.fna -strand plus -threads 32 $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH6/truncated_rv_FISH6.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH6/OUTPUT.uchime -nonchimeras usearch7.0/FISH6/nonchimeras_FISH6.fna -strand plus -threads 32 $ usearch7.0/usearch7.0.1090_i86linux32 -uchime_ref truncate_rv/FISH7/truncated_rv_FISH7.fna -db qiime.files/97_otus.fasta -uchimeout usearch7.0/FISH7/OUTPUT.uchime -nonchimeras usearch7.0/FISH7/nonchimeras_FISH7.fna -strand plus -threads 32 $ cat usearch7.0/FISH1/nonchimeras_FISH1.fna usearch7.0/FISH2/nonchimeras_FISH2.fna usearch7.0/FISH3/nonchimeras_FISH3.fna usearch7.0/FISH4/nonchimeras_FISH4.fna usearch7.0/FISH5/nonchimeras_FISH5.fna usearch7.0/FISH6/nonchimeras_FISH6.fna usearch7.0/FISH7/nonchimeras_FISH7.fna > merged_nonchimeras_FISH.fna $ pick_otus.py -i merged_nonchimeras_FISH.fna -r qiime.files/97_otus.fasta -o pick_otus/ --denovo_otu_id_prefix denovo_ -m uclust_ref --threads 32 $ pick_rep_set.py -i pick_otus/merged_nonchimeras_FISH_otus.txt -f merged_nonchimeras_FISH.fna -o rep_seqs_FISH.fna $ assign_taxonomy.py -i rep_seqs_FISH.fasta -m rdp -r 97_otus.fasta -t 97_otus_taxonomy.txt -o taxonomy60/ -c 0.6 $ align_seqs.py -i rep_seqs_FISH.fasta -t qiime_files/core_set_aligned.fasta.imputed -o pynast_align/ $ filter_alignment.py -i pynast_align/rep_seqs_FISH_aligned.fasta -m qiime_files/lanemask_in_1s_and_0s -o filtered_align/ $ make_phylogeny.py -i filtered_align/rep_set_FISH_aligned.fasta -o rep_seqs_FISH_tree.tre $ make_otu_table.py -i pick_otus/chimera_filtered_otus.txt -t assign_taxa/rep_seqs_tax_assignments.txt -o otu_table.biom -e pynast_aligned/failures.fasta $ filter_otus_from_otu_table.py -i otu_table.biom -o otu_table.filtered.biom -s 2 -n 5 $ single_rarefaction.py -i otu_table.filtered.biom -o otu_table.filtered.rarefied600.biom -d 600 $ alpha_diversity.py -i otu_table.filtered.rarefied600.biom -o alpha_div/ -m PD_whole_tree,observed_species,chao1,shannon,simpson_e -m FISH_mapping.txt -t rep_seqs_FISH_tree.tre $ beta_diversity.py -i otu_table.filtered.rarefied600.biom -m unweighted_unifrac,weighted_unifrac,bray_curtis -o beta_div/ -t rep_seqs_FISH_tree.tre $ principal_coordinates.py -i beta_div/ -o beta_coords/ $ make_2d_plots.py -i beta_coords/ -o 2d_plots/ -m FISH_mapping.txt $ summarize_taxa_through_plots.py -i otu_table.filtered.rarefied600.biom -o Tax_composition_FISH -m FISH_mapping.txt 3.2 picrust scripts $ pick_closed_reference_otus.py -i merged_nonchimeras_FISH.fna -r qiime.files/97_otus.fasta -t 97_otus_taxonomy.txt -o pick_closed_otus/ --assign_taxonomy (quality-filtered and chimeric sequence removed input fasta file was brought from precedent 16S rRNA gene sequence analysis) $ normalized_by_copy_number.py -i pick_closed_otus/otu_table.biom -o picrust/normalized_otus_table.biom $ predict_metagenomes.py -i picrust/normalized_otus_table.biom -o picrust/metagenome_predictions.biom -a pictust/nsti_per_sample.txt $ categorize_by_function.py -i picrust/metagenome_predictions.biom -o picrust/predicted_metagenomes.biom -l 3 -c KEGG_Pathways