U ¦ºeZ-ã@s¨ddlZddlZdddddejdddd d d d d ddddddhZdd„Zd-dd„Zd.dd„Zdd„Zd/dd„Z dd „Z d!d"„Z d#d$„Z d%d&„Z d'd(„Zd)d*„Zd+d,„ZdS)0éNzNot applicableÚUnknownZ UnspecifiedzMissing: Not collectedzMissing: Not providedzMissing: Not ProvidedÚmissingÚzMissing: Restricted accesszMissing:Not reportedzMissing: Not applicableZNAÚnullZNULLZno_dataÚNoneÚnanÚNaNcCsdddl}|dks|dkr"|s"|j}|dk rF|rFtjddd| d¡|dk rR|}tdd „|DƒƒS) z?In support of buffered: determine whether to use from_ or nargsrNú-z0Unable to handle --from as well as cmdline itemsT)Úerrécss|]}| ¡VqdS©N)Ústrip)Ú.0Ús©rúk/mounts/lovelace/software/anaconda3/envs/qiime2-amplicon-2023.9/lib/python3.8/site-packages/redbiom/util.pyÚ sz from_or_nargs..)ÚsysÚstdinÚclickZechoÚexitÚiter)Zfrom_Znargs_variablerrrrÚ from_or_nargss ÿ rr csøddl}ddl}ddl}| ¡}|j |¡}tƒ}|dkrFtd|ƒ‚t|t tt fƒs\|g}‡fdd„} t |ƒ}|j j   d|¡} |D]l} d} |D]P} | || d| | ƒƒ}|sÆ| dkrºtƒ} |  |¡q’| dkrØt|ƒ} q’|  |¡} q’| r†| | ¡}q†|S)a²Grab samples from an iterable of IDs Parameters ---------- it : iteraable of str The IDs to search for exact : boolean If True, compute the intersection of results per context. If False, compute the union of results per context. axis : {'feature', 'sample'} The axis to operate over. contexts : list of str The contexts to search in min_count : int, optional The minimum count (inclusive) to retain an observation. Notes ----- Contexts are evaluated independently, and the results of each context are unioned. Returns ------- set The IDs associated with the search IDs. rN>ÚsampleÚfeaturezUnknown axis: %scs‡fdd„| ¡DƒS)Ncsi|]\}}|ˆkr||“qSrr)rÚkÚv©Ú min_countrrÚ Msz6ids_from..min_count_filter..)Úitems)ZdatrrrÚmin_count_filterLsz"ids_from..min_count_filterzfetch-%s)ÚredbiomÚredbiom._requestsZ redbiom.adminÚ get_configÚ _requestsZmake_script_execÚsetÚ ValueErrorÚ isinstanceÚlistÚtupleZadminZ ScriptManagerÚgetÚupdateÚ intersectionÚunion)ÚitÚexactÚaxisZcontextsrr"ÚconfigÚseZ retrievedr!ZfetcherÚcontextZ context_idsZid_ÚblockrrrÚids_from"s6       r6cCs<|dkr,ddl}ddl}| ¡}|j |¡}|ddd|ƒS)a^Test if a category exists Parameters ---------- category : str The category to test for get : function A get method Returns ------- bool True if the category exists, False otherwise Redis Command Summary --------------------- SISMEMBER metadata:catetories-represented NrÚmetadataZ SISMEMBERzcategories-represented/%s)r"r#r$r%Úmake_get)Úcategoryr+r"r2rrrÚcategory_existses  r:cCs*z t|ƒWStk r$tjYSXdSr )ÚfloatÚ ExceptionÚnpr)ÚtrrrÚ float_or_nanƒs r?c Cstddl}|dkr,ddl}| ¡}|j |¡}t|ƒ\}}}}t|dddƒƒ}t|ƒ |¡s^dSt|ƒ |¡spdSdS)z(Test if all samples have sample metadatarNr7ÚSMEMBERSúsamples-representedFT)r#r"r$r%r8Úpartition_samples_by_tagsr&Úissubset) Úsamplesr+r"r2ÚuntaggedÚtaggedÚ_Ú tagged_cleanZ representedrrrÚhas_sample_metadataŠs rIc Cspg}g}g}g}|D]N}| dd¡}t|ƒdkrX|\}}| |¡| |¡| |¡q| |¡q||||fS)z1Partition samples by the presence of a sample tagrGr é)ÚsplitÚlenÚappend) rDrFrHÚtagsrErÚpartsÚtagZ sample_splitrrrrBžs      rBcCs(ddlm}t|ƒ\}}}}||ddƒ}t|ƒ\}} }} |tƒ} t| | ƒD]\} } | |  | ¡qJt| ƒ}g}i}|D]$}|| kr| |||<qv| |¡qvt|ƒ\}}g}t||ƒD]D\}}||krð| |¡||kràg||<|| |¡q¶| |¡q¶t|ƒ\}}|  |¡|  |¡||||fS)aÑDetermine mappings for requested samples This method accepts samples in the form of "sampleid" or "rid_sampleid". It then attempts to resolve any sample ambiguities which may exist in the context. For a "sampleid" there may be multiple "rid_sampleid" values which exists, for instance, the same sample may have multiple preps within the same study and datatype (e.g., biological replicates). Parameters ---------- context : str The context to search within samples : Iterable of str The samples to resolve. The samples must be in the form of "sampleid" or "rid_sampleid". The former is an ambiguous association as it does not have a tag affixed (e.g., a sample preparation ID). The latter is fully specified and assured to be unique within the context. get : redbiom._requests.make_get instance A getter Returns ------- stable A dict of stable QIIME compatible sample IDs, keyed by the QIIME compatible ID and valued by the redbiom ID. unobserved A list of any requested ID which was not observed in the context. ambituities A dict of untagged sample IDs (e.g., "sampleid") to a list of the observed "rid_sampleid" values within the context. In other words, this dict associated an unspecific ID to a unique redbiom ID. redbiomids A dict keyed by "rid_sampleid" and valued by a QIIME compatible sample ID. r©Ú defaultdictr@rA) Ú collectionsrRrBr)ÚziprMr&Ú_stable_ids_from_ambigÚ_stable_ids_from_unambigr,)r4rDr+rRrErFrGrHÚctxZ ctx_taggedZctx_tagged_cleanZctx_with_ambigZwith_tagZ without_tagZctx_known_stableZ unobservedZknown_ambiguousÚiZstableÚriZ unambiguousr>ZtcZ stable_unambZri_unambrrrÚresolve_ambiguities²s6$         rZc CsXi}i}| ¡D]>\}}|D]0}| dd¡\}}d||f}|||<|||<qq||fS)z'Create stable IDs from an ambiguity maprGr ú%s.%s)r rK) Z ambig_mapZ ambig_assocrYrrÚunambigrPrEÚstabrrrrUs  rUcCsFi}i}|D]0}| dd¡\}}d||f}|||<|||<q ||fS)z)Create stable IDs from an unambiguous IDsrGr r[)rK)r\ÚassocrYrrPrEr]rrrrVs  rVcCsäddlm}m}ddlm}ddl}ddl}| |jj¡}||t ƒddƒ}|j j d|krl|g|j j |j _ t |j j d¡ƒ}| t||¡} |tƒ} | ¡D]>\} } | |  ¡} | jD] } | | ƒD]}| | | ¡qÄq¸qœt| ƒS)zÒConvert a DataFrame to stem -> index associations Parameters ---------- df : pd.DataFrame A pandas DataFrame to index Returns ------- dict {stem: {set of indices}} r©ÚjoinÚdirnamerQNÚassetsÚ nltk_dataÚenglish)Úos.pathr`rarSrRÚ functoolsÚnltkZ PorterStemmerZMARTIN_EXTENSIONSÚ__file__ÚdataÚpathÚ frozensetZcorpusZ stopwordsÚwordsÚpartialÚstemsr&ZiterrowsZisnullÚvaluesÚaddÚdict)Údfr`rarRrfrgÚstemmerÚnltk_data_pathÚstopsZstem_fÚdrÚrowÚvalueÚstemrrrÚ df_to_stems$s"    rzc csddlm}m}ddl}ddl}tdƒ}| t¡| d¡}| d¡} ||krRdS||t ƒddƒ} |j j d| kr„| g|j j |j _ |j   |¡D]|} | |kst| ƒd krªq| |ksd | kr¼q| | ¡dk rÌq|  | ¡dk rÜqz| | ¡ ¡VWqtk r YqYqXqdS) zGather stems from stringrr_Nz()!@#$%^&*-+=|{}[]<>./?;:z(^-?\d+\.\d+$)|(^-?\d+$)z^\d+:\d+(am|AM|pm|PM)?$rbrcr ú/)rer`raÚrergr&r,Ú NULL_VALUESÚcompilerhrirjÚtokenizeZ word_tokenizerLÚmatchryÚlowerr<) rursÚstringr`rar|rgZto_skipZ numeric_regexZ time_regexrtÚwordrrrrnJs2   rncCsZddl}tƒ}|jddD](}| ¡}| | d¡¡| ¡||<q|j|dd||fS)zÙconvert biom feature ids to md5 and return new table Parameters ---------- table : BIOM table Returns ------- BIOM table The new BIOM table dict {original_id: new_id} rNZ observation)r1zutf-8)ÚhashlibrqZidsÚmd5r,ÚencodeÚ hexdigestZ update_ids)Útabler„Znew_idsZ_idÚmrrrÚconvert_biom_ids_to_md5usrŠ)r )N)N)rÚnumpyr=rr}rr6r:r?rIrBrZrUrVrzrnrŠrrrrÚs@ø  C  P&+