U _j=@s<ddlZddlZddlZddlZddlZddlZddlZddlm Z ddlm Z ddlZddlm Z ddlm Z ddl mZejejeZddZd d Zd d Zd2ddZddZddZd3ddZd4ddZd5ddZddZddZdd Zd6d"d#Z d7d$d%Z!d&d'Z"d(d)Z#d8d+d,Z$d-d.Z%d9d0d1Z&dS):N) constants)bins) gffwriter)parser) dict_classcCstjtdd|S)zG Return the full path of a data file that ships with gffutils. Ztestdata)ospathjoinHERE)fnr /lib/python3.8/site-packages/gffutils/helpers.pyexample_filenamesrcCs(t|tjr|g}dd|D}t|S)a! Infer the dialect based on the attributes. Parameters ---------- attributes : str or iterable A single attributes string from a GTF or GFF line, or an iterable of such strings. Returns ------- Dictionary representing the inferred dialect cSsg|]}t|dqS))rZ_split_keyvals.0ir r r )sz!infer_dialect..) isinstancesix string_types_choose_dialect) attributesdialectsr r r infer_dialects rcCsTt|dkrtjSg}|D]$}|dD]}||kr&||q&q|d}||d<|S)a Given a list of dialects, choose the one to use as the "canonical" version. If `dialects` is an empty list, then use the default GFF3 dialect Parameters ---------- dialects : iterable iterable of dialect dictionaries Returns ------- dict rorder)lenrdialectappend)rZ final_orderror r rr-s  rFc  Csd} ttjddddddd} |r(|| d<|r4|| d<| d| dd} t|| krbtd||rt|tjrd| d <| |n&d d d d |D| d <| ||rdt|tjr| d\} } | d\}}n |\} }}t j t|t|dd}|rd| d<| | ||gnd| d<| | ||gt|dkrd| ddd tt|7<|r|d| d<| |tjddg}g}|rt|tjr| |n>|D]8}||krtd||f|dkrd}| |qd |}|rd}nd}d||f| d <d}d!| dkr0d"}d#D]<}| |r4|s^d$| || |<d"}nd%| || |<q4| jf| |fS)&a{ Multi-purpose, bare-bones ORM function. This function composes queries given some commonly-used kwargs that can be passed to FeatureDB methods (like .parents(), .children(), .all_features(), .features_of_type()). It handles, in one place, things like restricting to featuretype, limiting to a genomic range, limiting to one strand, or returning results ordered by different criteria. Additional filtering/subsetting/sorting behavior should be added here. (Note: this ended up having better performance (and flexibility) than sqlalchemy) This function also provides support for additional JOINs etc (supplied via the `other` kwarg) and extra conditional clauses (`extra` kwarg). See the `_QUERY` var below for the order in which they are used. For example, FeatureDB._relation uses `other` to supply the JOIN substatment, and that same method also uses `extra` to supply the "relations.level = ?" substatment (see the source for FeatureDB._relation for more details). `args` contains the arguments that will ultimately be supplied to the sqlite3.connection.execute function. It may be further populated below -- for example, if strand="+", then the query will include a strand clause, and the strand will be appended to the args. `args` can be pre-filled with args that are passed to `other` and `extra`. zC{_SELECT} {OTHER} {EXTRA} {FEATURETYPE} {LIMIT} {STRAND} {ORDER_BY})_SELECTOTHER FEATURETYPELIMITSTRANDORDER_BYEXTRAr#r(?z!Not enough args (%s) for subqueryzfeatures.featuretype = ?r$zfeatures.featuretype IN (%s),cSsg|]}dqS)r)r )r_r r rrszmake_query..:-FZonez@features.seqid = ? AND features.start >= ? AND features.end <= ?r%z@features.seqid = ? AND features.start <= ? AND features.end >= ?iz AND features.bin IN (%s)zfeatures.strand = ?r&Z file_orderlengthz#%s not a valid order-by value in %sz (end - start)ZDESCZASCzORDER BY %s %sr'whereT)r(r$r%r&zWHERE zAND )dictrr"countr ValueErrorrrrrr extendsplitrintmapstrZ_gffkeys_extralowerformat)argsotherlimitZstrand featuretypeextraZorder_byreverseZcompletely_withinZ_QUERYdZ required_argsZseqidZ startstopstartendZ_binsZvalid_order_byZ _order_byk directionr0rr r r make_queryNs"               rFcCsFz*t|d}t|d}tj||ddWStk r@YdSXdS)zQ Given a dictionary yielded by the parser, return the genomic "UCSC" bin rBrCTr.N)r6rr3)rArBrCr r r_bin_from_dicts   rGcCs(t|trtj|jddStj|ddS)zUse most compact form of JSON)r*r,)Z separators)rrjsondumpsZ_dxr r r_jsonifys rLcCs |rt|}t|St|S)z.Convert JSON string to an ordered defaultdict.)rHloadsr)rKZ isattributesobjr r r _unjsonifys rOTcCsHg}tjD]4}t||}|r4|dkr4|t|q ||q t|S)z= Convert feature to tuple, for faster sqlite3 import rr?)r_keysgetattrrrLtuple)fjsonifyrKrDvr r r_feature_to_fieldss    rWcCsFg}tjD]2}||}|r2|dkr2|t|q ||q t|S)z: Convert dict to tuple, for faster sqlite3 import rP)rrQrrLrS)rArUrKrDrVr r r_dict_to_fieldss   rXcCsddl}|t|dS)z> Converts a gffutils.Feature to a pybedtools.Interval rN ) pybedtoolsZcreate_interval_from_listr8r5)ZfeaturerZr r r asinterval+sr[cCst|}|t||D]\}}t|ts"|g||<q"t|D].\}}||krJt|tsj|g}|||qJt dd|DS)z Merges two attribute dictionaries into a single dictionary. Parameters ---------- `attr1`, `attr2` : dict Returns ------- dict css"|]\}}|tt|fVqdSN)sortedset)rrDrVr r r Msz#merge_attributes..) copydeepcopyupdateitemsrlistrZ iteritemsr4r1)Zattr1Zattr2Znew_drDrVr r rmerge_attributes3s    recCs@t|}t|}ttt||tt||dS)z Compares two dialects. )ZaddedZremoved)r^rcr1rd difference)Zdialect1Zdialect2Zorignewr r rdialect_comparePs   rhgidcs$fdd}tj|ddd}|S)a Sanitize given GFF db. Returns a sanitized GFF db. Sanitizing means: - Ensuring that start < stop for all features - Standardizing gene units by adding a 'gid' attribute that makes the file grep-able TODO: Do something with negative coordinates? c3sVD]H}|dj}|D]4}|j|jkr<|j|j|_|_|g|j<|VqqdS)Nr)Ziter_by_parent_childsidrBstopr)Z gene_recsZgene_idZrecdb gid_fieldr rsanitized_iteratorhs    z+sanitize_gff_db..sanitized_iterator:memory:Fverbose)gffutils create_db)rmrnro sanitized_dbr rlrsanitize_gff_db\s  rvcCsd}t|rt|}n|r.tj|ddd}nt|}|rJtj||d}n ttj}t |}|j ddD]}| ||j qj| dS)z Sanitize a GFF file. NrpFrq)in_placegene)r>) is_gff_dbrsZ FeatureDBrt get_gff_dbrZ GFFWritersysstdoutrvZ all_featuresZwrite_gene_recsrjclose) gff_fnameZ in_memoryrwrmZgff_outruZgene_recr r rsanitize_gff_file{s"  rcCsdS)zq Annotate a GFF file by cross-referencing it with another GFF file, e.g. one containing gene models. Nr )rmr r rannotate_gff_dbsrcCs"tj|sdS|drdSdS)zc Return True if the given filename is a GFF database. For now, rely on .db extension. F.dbT)rr isfileendswith)db_fnamer r rrys   ryutf-8cCs(t|tjr$t|tjs$t||}|Sr\)rrrZ text_type)rNencodingr r r to_unicodes   rc#sddl}|j|dd|dD]}g}t|j|ddD]`\}}d}d}t|j|dd} | D]&} t| } | jdkr~|| 7}|| 7}q`||||| fq:t dd |Ddkrt |d} nt |d d d} t | | d } | d }fdd| D}|d |fVq dS)NrT)Zas_rawrxr)levelZCDScss|]}|dVqdS)rNr rr r rr_sz(canonical_transcripts..cSs|dS)Nrr rJr r rz'canonical_transcripts..csg|]}|qSr )ZsequencerZfastar rrsz)canonical_transcripts..r!) pyfaidxZFastaZfeatures_of_type enumerateZchildrenrdrr>rmaxr]printr )rmZfasta_filenamerrxZ exon_listZtiZ transcriptZcds_lenZ total_lenZexonsZexonZ exon_lengthZbestZcanonical_exonsZseqsr rrcanonical_transcriptss,  rrcCstj|std|d||f}tj|r4|Stjdd}td|t}tj ||j ddd}t}td|||S) z Get db for GFF file. If the database has a .db file, load that. Otherwise, create a named temporary file, serialize the db to that, and return the loaded database. zGFF %s does not exist.z%s.%sF)deletezCreating db for %smerge)Zmerge_strategyrrz - Took %.2f seconds) rr rr3tempfileZNamedTemporaryFilertimersrtname)r~extZcandidate_db_fnamert1rmt2r r rrzs       rz)NNNNNNFF)F)T)T)ri)TF)r)r)'r`r{rZ simplejsonrHrrrrsrrrrZgffutils.attributesrr dirnameabspath__file__r rrrrFrGrLrOrWrXr[rerhrvrrryrrrzr r r rsR     ! %      )