U L_q&@sdZddlZddlZddlmZddlmZddlmZddl m Z m Z ddl Zddl m Z ddlmZdd lmZmZeeZGd d d eZGd d d eZddZddZedkredS)z, Created on May 31, 2015 @author: namphuon N) get_logger)MutableAlignment)ExhaustiveAlgorithm)optionsvalid_decomp_strategy)lcm) SeppProblem)JobPoolJoinc@s0eZdZdZddZddZddZdd Zd S) EnsembleJoinSearchJobsz After all search jobs have finished on tips, we need return the distribution of the bitscores for the search. This join accomplishes this cCst|dSN)r __init__selfr,lib/python3.8/site-packages/sepp/ensemble.pyr szEnsembleJoinSearchJobs.__init__cCs(||_|D]}||jdqdS)N hmmsearch) root_problem iter_leavesZadd_jobjobs)rrprrrsetup_with_root_problems z.EnsembleJoinSearchJobs.setup_with_root_problemcCs tddS)zP A dummy join that waits for all the search results to complete N)printrrrrperform!szEnsembleJoinSearchJobs.performcCs d|jfS)Nz!join search jobs for all tips of )rrrrr__str__'szEnsembleJoinSearchJobs.__str__N)__name__ __module__ __qualname____doc__r rrrrrrrr s r c@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ dS)EnsembleExhaustiveAlgorithmz This implements the exhaustive algorithm where all alignments subsets are searched for every fragment. This is for UPP, meaning that no placement is performed, and that there is always only one placement subset (currently). cCs t|d|_d|_d|_dS)NFT)rr ZsymfracZelimfiltersrrrrr 2s z$EnsembleExhaustiveAlgorithm.__init__cCsVdt_tjdks tjdkr2tdtdt}|t |jjj t |S)NZ A_dummy_valuez;Specify the backbone alignment and tree and query sequences) r info_file tree_filealignment_file_LOGerrorexitrZread_file_objectopennamer check_options)r sequencesrrrr+8sz)EnsembleExhaustiveAlgorithm.check_optionscCs8|jjdks |jj|ks tdt||||j_dS)Nzfcurrently eHMMs works with only one placement subset. Please leave placement subset size option blank.)rplacement_sizeAssertionErrorrcheck_and_set_sizes)rZtotalrrrr/Cs   z/EnsembleExhaustiveAlgorithm.check_and_set_sizesc Csd|jjkrdStdd|jjD}|jD]j}|}t|tsNt |jdkrh|jj g|_| d}| D]"\}}|| |d|dgqzq4g}| D]\}}t|dkr| |qd|jjd<tdt|||_dS) z merges search resultszfragments.distribution.doneNcSsg|] }|gfqSrr).0r*rrr Osz=EnsembleExhaustiveAlgorithm.merge_results..rrz.Fragments %s are not scored against any subset)rZ annotationsdictZ fragmentskeysrZ get_parent isinstancerr.Zget_soft_sub_alignmentZget_job_result_by_nameitemsappendlenr&Zwarningstrresults) rZsequence_scoresZfragment_chunk_problemZ align_problemZ search_reskeyvalZ notScoredvrrr merge_resultsKs2       z)EnsembleExhaustiveAlgorithm.merge_resultsc sddt|jtst|jD]V}|jD]J}t|ts>t|jd}|D]$}|jd}||ffdd qPq,q"t}| |jdS)z< a callback function called after hmmbuild jobs are finishedcSs||_t|dSr )Zhmmmodelr Z enqueue_job)resultZ search_jobrrrenq_job_searchfragmentuszHEnsembleExhaustiveAlgorithm.connect_jobs..enq_job_searchfragmentZhmmbuildrcs ||Sr r)r?Znext_jobr@rrsz:EnsembleExhaustiveAlgorithm.connect_jobs..N) r5rrr.Z get_childrenZchildrenrZ add_call_Backr r)rZplacement_problemZ alg_problemZbjZ fc_problemZsjZjsjrrAr connect_jobsss     z(EnsembleExhaustiveAlgorithm.connect_jobsc Cs|j}td|d}|d}t|d}t|d}|d|D]P\}}t|dkrn|d|qJ|D]&}|d||dd |d fqrqJ| | dS) Nz"Generating csv of search results. z scores.csvz unmatched.csvwzseq,bitscore,evalue rz%s z %s,%0.4f,%s z{:.3e}r2) r:r&infoZget_output_filenamer)writer6r8formatclose) rZsearch_resultsZ outfilenameZ not_matchedfZ unmatchedr;valueZpairrrroutput_resultss$        z*EnsembleExhaustiveAlgorithm.output_resultscCs.tt|j}t||jj|}||Sr )r8listrrrrZcpuZread_and_divide_fragments)rZalg_subset_countZfrag_chunk_countrrrcreate_fragment_filessz1EnsembleExhaustiveAlgorithm.create_fragment_filesN) rrrrr r+r/r>rCrKrMrrrrr +s (r c Csbtjtjdtj}d|_|jd}d ddg|j d<|j dd t d d d d d|j ddt ddddd|jd}|j ddtdddddd|j ddddtdd d!|j d"d#d$d%tdd&d!|j d'd(d)d*tdd+d!|d,d-|d.d/}|j d0d1t d2d dd3d|j d4d5d6d7tdd8d!|j d9d:d;dNz~/.sepp/upp.configzThis script runs the UPP algorithm on set of sequences. A backbone alignment and tree can be given as input. If none is provided, a backbone will be automatically generated. decompGroup z These optionszadetermine the alignment decomposition size, backbone size, and how to decompose the backbone set. descriptionz-Az--alignmentSizeZalignment_sizeN z,max alignment subset size of N [default: 10])typedestmetavardefaulthelpz-Sz--decomp_strategyZdecomp_strategyZDECOMPZ hierarchicalzAdecomposition strategy [default: ensemble of HMMs (hierarchical)] inputGroupz-sz--sequence_filerZ sequence_fileZSEQzUnaligned sequence file. If no backbone tree and alignment is given, the sequence file will be randomly split into a backbone set (size set to B) and query set (remaining sequences), [default: None]z-cz--configZ config_fileZCONFIGzA config file, including options used to run UPP. Options provided as command line arguments overwrite config file values for those options. [default: %(default)s])rTrUrSrWz-tz--treer$ZTREEz6Input tree file (newick format) [default: %(default)s]z-az --alignmentr%ZALIGNz)Aligned fasta file [default: %(default)s]z UPP Optionsz*These options set settings specific to UPPz SEPP OptionszEThese options set settings specific to SEPP and are not used for UPP.z-Pz--placementSizer-zJmax placement subset size of N [default: 10%% of the total number of taxa]z-rz--raxmlr#ZRAXMLzURAxML_info file including model parameters, generated by RAxML.[default: %(default)s]z-fz --fragmentZ fragment_fileZFRAGz$fragment file [default: %(default)s])seppZconfigZset_main_config_pathospath expanduserZ get_parserrPgroupsjoin__dict__ add_argumentintrargparseZFileTypeZadd_argument_groupupper)parserrNrXZ seppGrouprrraugment_parsers    rfcCsttdSr )rfr runrrrrmainsrh__main__)rrcr[rZrZsepp.alignmentrZsepp.exhaustiverZ sepp.configrrZsepp.math_utilsrZ sepp.problemrZsepp.schedulerr r rr&r r rfrhrrrrs"     xS