чoeYBddlZddlZddlZddlmZmZddlmZddl m Z m Z ddl Z ddl mZdZdZgdZd Zd Zd Zd Zeeeeeed ZdZddhZddhZhdZhdZdZdZ dZ!dZ"dZ#dZ$dZ%dZ&dZ'd,d Z(d!Z)Gd"d#e*Z+d$Z,d%Z-d-d'Z. d.d(Z/d)Z0d/d+Z1dS)0N)BiomParseExceptionUnknownAxisError)Table) biom_open __version__) defaultdictJustin Kuczynskiz5Copyright 2011-2020, The BIOM Format Development Team)r Daniel McDonaldz Greg CaporasozJose Carlos Clemente LitranzAdam Robbins-PiankazJose Antonio Navas MolinaBSDzhttp://biom-format.orgr zdaniel.mcdonald@colorado.edu)intfloatunicoder r r"[{]}>   ,> 0123456789rrrc|d|z}|dkrdS|t|zdz}|}||tvr|dz }||tv||tvr||dvr|dz }||dvn||g}|dz }|r||}|tkr<|dtkr|nh||nR|tvr+ |n3#t$r|dz}Yn)wxYw|tvr|||dz }||||S)z|Returns key:value from the biom string, or "" This method pulls an arbitrary key/value pair out from a BIOM string z"%s":)rrr) findlen JSON_START JSON_OPENQUOTEpopappend JSON_CLOSE IndexError)biom_strkeybase_idx start_idxcur_idxstackcur_chars _/mounts/lovelace/software/anaconda3/envs/kraken-biom/lib/python3.11/site-packages/biom/parse.pydirect_parse_keyr89s }}Ws]++H2~~rs3xx'!+ G 7 : - -1  7 : - - ))w66 qLGw66 '"#1  (H5  9%%IIKKKKLL****Z''IIKKKK!qLGEY&& X&&& qLG! $ HW$ %%s-DDDc&|dvrtdt|d}|dkrtdt|d}|dkrtdt|d}|dkrtd|dd d dd d}t t t|d \}}|d d z} || t|d z }t|dkrtdd} |dkr7t||krtd| t||fz} n<|dkr6t||krtd| |t|fz} t|}g} |dkrt||} n|dkrt||} d| d| S)a0Pull out specific slices from a BIOM string biom_str : JSON-formatted BIOM string to_keep : indices to keep axis : either 'samples' or 'observations' Will raise IndexError if the inices are out of bounds. Fully zerod rows or columns are possible and this is _not_ checked.  observationsamplezUnknown axis typeshaper$.biom_str does not appear to be in BIOM format!data matrix_type:r#rrrr&rz'Observations to keep are out of bounds!z[%d, %d]r;r<z"Samples to keep are out of bounds!z"data": z , "shape": )r/r8 ValueErrorsplitreplacelistmapr r'r(minmaxset_direct_slice_data_sparse_obs_direct_slice_data_sparse_samp) r0to_keepaxis shape_kv_pair data_fieldsmatrix_type_kv_pair raw_shapen_rowsn_cols data_start new_shapenew_datas r7direct_slice_datarWgs; ,,,,---%Xw77MIJJJ"8V44KbIJJJ*8]CCb  IJJJ##C((,44S"==EEc2NNI#c9??3#7#78899NFF!!#&&*Jj[)9)9A)==>K 7||aBCCCI } w<<6 ! !FGG GWv 66    w<<6 ! !ABB BW 66 'llGH }0gFF   1+wGG 6h 6 69 6 66c,|dS)Nz[] stripxs r7strip_fr^s 779  rXcttt|d\}}}||d|d|S)zRemap a sparse observation axisrrErFr^rCrcvlookuprowcolvalues r7_remap_axis_sparse_obsrgsI3w #7788OCeSk ) )C ) )% ) ))rXcttt|d\}}}|d||d|S)zRemap a sparse sample axisrr`ras r7_remap_axis_sparse_samprisI3w #7788OCe ) )F3K ) )% ) ))rXcLg}dtt|D}|dD]O}t|d\}}}||vr#|t ||Pdd|zS)zislice observations from data data : raw data string from a biom file to_keep : rows to keep c4i|]\}}t||Sstr.0ivs r7 z1_direct_slice_data_sparse_obs..$EEE$!QCFFAEEErX],r[[%s]]],[) enumeratesortedrCr^r-rgjoinr?rLrV remap_lookuprbrcrrs r7rJrJsHEE)F7OO*D*DEEELzz$GG#,,$$S))1a   OO23 EE F F F ejj** **rXc2g}dtt|D}|dD]B}|d\}}}||vr#|t ||Cdd|zS)zgslice samples from data data : raw data string from a biom file to_keep : columns to keep c4i|]\}}t||Srlrmros r7rsz2_direct_slice_data_sparse_samp..rtrXrurrvrw)rxryrCr-rirzr{s r7rKrKsHEE)F7OO*D*DEEELzz$HH))C..1a   OO3CFF G G G ejj** **rXct|dkrd}t||}n(|dkrd}t||}ntd|dkrtdtjd|z}d ||D}|st d fd t||D}t|}|gi}t||D]$\} } | |vr||| %|tj |d d fS)zReturns the indices for the associated ids to keep biom_str : a BIOM formatted JSON string to_keep : a list of IDs to get indices for axis : either 'samples' or 'observations' Raises KeyError if unknown key is specified r;rowsr<columnsz Unknown axis!r$r>z{%s}ch|] }|d Sidrl)rprrs r7 z#get_axis_indices..s4441qw444rXz+Not all of the to_keep ids are in biom_str!c.g|]\}}|dv|Srrl)rprqrrrLs r7 z$get_axis_indices..s+ O O O$!QAdGw_``. Everything before the first underscore will be used as the sample identifier in the resulting ``Table``. The information after the first underscore is not used directly, though the full identifiers of seeds will be used as the observation identifier in the resulting ``Table``. HSLrr *HS_zA query sequence was encountered that does not have an underscore. An underscore is required in all query sequence identifiers to indicate the sample identifier.Nr&)observation_ids sample_ids) rr rIr[rCr(r-rindexrBr)fhr? sample_idxsrobservation_idxsr line_typeslinefields line_typeobservation_idquery_idobservation_idxunderscore_index sample_id sample_idxs r7parse_ucrs: s  DKJOUJ33zz||  D!!1I J & &  **1-!9??$$Q' S %N - - -.~>OO!/22O  " "> 2 2 2/> ^ ,   yC// L#+??3#7#7   L L L KLLL L!!2"2!23IK''(3  __ )3 I&!!),,, /:. / / /1 4 / / / /  : N N NNs 5D  D%r<Fc,|dvrt| tj||S#t$rYnt$rYnwxYwt |dr|}|d}|r)|d}|)|dkr>| |tj tj ||}n| |tj |ddd}nt|tre tj tjd ||}nP#t$rtj |ddd }Yn,wxYwtj tj||}fd }d }8||| |dkrdnd}||| |S)aParses the biom table stored in `file_obj` Parameters ---------- file_obj : file-like object, or list file-like object storing the BIOM table (tab-delimited or JSON), or a list of lines of the BIOM table in tab-delimited or JSON format ids : iterable The sample/observation ids of the samples/observations that we need to retrieve from the biom table axis : {'sample', 'observation'}, optional The axis to subset on input_is_dense : boolean Indicates if the BIOM table is dense or sparse. Valid only for JSON tables. Returns ------- Table The BIOM table stored at file_obj Raises ------ ValueError If `samples` and `observations` are provided. Notes ----- Subsetting from the BIOM table is only supported in one axis Examples -------- Parse a hdf5 biom table >>> from h5py import File # doctest: +SKIP >>> from biom.parse import parse_biom_table >>> f = File('rich_sparse_otu_table_hdf5.biom') # doctest: +SKIP >>> t = parse_biom_table(f) # doctest: +SKIP Parse a hdf5 biom table subsetting observations >>> from h5py import File # doctest: +SKIP >>> from biom.parse import parse_biom_table >>> f = File('rich_sparse_otu_table_hdf5.biom') # doctest: +SKIP >>> t = parse_biom_table(f, ids=["GG_OTU_1"], ... axis='observation') # doctest: +SKIP r:)idsrMreadr&r)input_is_denseNc|SNrlr\s r7z"parse_biom_table..qrXr$c|Srrlr\s r7rz"parse_biom_table..rrXc |vSrrl)r?id_mdrs r7 subset_idsz$parse_biom_table..subset_idss czrXc*tj|Sr)npany)valsrrs r7gt_zeroz!parse_biom_table..gt_zerosvd||rXrMr<r;)rr from_hdf5rB RuntimeErrorhasattrtellrisspaceseek from_jsonrloadfrom_tsv isinstancerErrzfilter) file_objrrMrold_posr~trrs ` r7parse_biom_tablerYsf^ ,,, xSt<<<<           x  ;--// MM!  iikk ! a  Aiikk ! 88 MM' " " " ( 3 3/=???AA MM' " " "xt[[AAAA Hd # #; B 2778+<+< = =/=???AA B B Bxt[[AAAAA B ODJx00+9 ; ; ;  $''' $ 0 0}}h t$$$ Hs&- A AA;F"F'&F'cg}|dD]X}g}|dD])}||*||Y|S)N|;)rCr-r[)r]complex_metadataysimple_metadataes r7sc_pipe_separatedrs{ WWS\\11 . .A  " "17799 - - - -0000 rXc:eZdZe ddZfdZxZS) MetadataMapTFNc t|dr. t|}n#t$rtdwxYw|r |rd}n d}n |rd}nd}|i}g}|pg}g}|D]!} || } | r|r| s'| drJ| d d} |s(| d }p|| tt|| d } t| t|kr6| d gt|t| z z|| #|std |std d|D} t| tt| krtdi} |D]_} i}t|d d| d dD]/\}} |||||<#t$r|||<Y,wxYw|| | d<`|| S)a,Parse mapping file that relates samples or observations to metadata. Format: header line with fields optionally other comment lines starting with # tab-delimited fields process_fns: a dictionary of functions to apply to metadata categories. the keys should be the column headings, and the values should be functions which take a single value. For example, if the values in a column called "taxonomy" should be split on semi-colons before being added as metadata, and all other columns should be left as-is, process_fns should be: {'taxonomy': lambda x: x.split(';')} Assumes the first column in the mapping file is the id. This method is ported from QIIME (http://www.qiime.org), previously named parse_mapping_file/parse_mapping_file_to_dict. QIIME is a GPL project, but we obtained permission from the authors of this method to port it to the BIOM Format project (and keep it under BIOM's BSD license). upperzAA string was passed that doesn't refer to an accessible filepath.c.|ddSNrr$)rDr\s r7r^z&MetadataMap.from_file..strip_fs99S"---rXcR|ddSrrDr[r\s r7r^z&MetadataMap.from_file..strip_fs"99S"--33555rXc|Srrlr\s r7r^z&MetadataMap.from_file..strip_fsHrXc*|SrrZr\s r7r^z&MetadataMap.from_file..strip_fs7799$rXN#r&rr$z)No header line was found in mapping file.zNo data found in mapping file.cg|] }|d S)rrl)rprqs r7rz)MetadataMap.from_file..s000aQqT000rXz2First column values are not unique! Cannot be ids.r)ropenOSErrorrr[ startswithrCr-rErFr(extendrIzipr)clslines strip_quotessuppress_strippingheader process_fnsr^ mapping_datacommentsrtmp_line first_colmappingr current_dkrrs r7 from_filezMetadataMap.from_files22 5' " " M MU  M M M(*LMMM M  %! 6....6666" %%%%  K 2 . .D74==D . tzz|| s## .ABBx*!ZZ\\//55FFOOD)))) GTZZ-=-= > >??x==3v;;..OORDCKK#h--,G$HIII##H---- .$&-.. . G$%EFF F00<000 y>>SY00 0 0$&677 7  ) )DIF122JQRR11 % %1%#1;q>!#4#4IaLL%%%#$IaLLL%(GDG  s7||s"<H22IIcJt|dS)zAccepts dictionary mapping IDs to metadata. ``mapping`` should be a dictionary mapping an ID to a dictionary of metadata. For example: {'Sample1': {'Treatment': 'Fast'}, 'Sample2': {'Treatment': 'Control'}} N)super__init__)selfr __class__s r7rzMetadataMap.__init__,s# !!!!!rX)TFNN)__name__ __module__ __qualname__ classmethodrr __classcell__)rs@r7rrsaDI+/fff[fP"""""""""rXrcdtzS)zReturns a generated by stringzBIOM-Format %s)rrlrXr7 generatedbyr7s k ))rXc ntj||||fi|}|tS)aConvert a contigency table to a biom table sample_mapping : dict of {'sample_id':metadata} or None obs_mapping : dict of {'obs_id':metadata} or None process_func: a function to transform observation metadata dtype : type of table data )rrto_jsonr)table_fsample_mapping obs_mapping process_funckwargs otu_tables r7convert_table_to_biomr <sCw ^+77/577I   []] + ++rXrAct|tr|dSt|trtddgt|dtrRg}|D]8}fd|D}|d|9d|Sdfd|DSd S) zCDetermine which format the metadata is and then convert to a stringrz;|r$rc^g|])}|*Srl)r[trans)rprtranstabs r7rz'biom_meta_to_string..Ys-???!''))//(33???rXz; rc3hK|],}|dV-dS)rNr)rpr] replace_strs r7 z&biom_meta_to_string..`sQ--))C55;;==------rXN)rrnrDrEbytes maketransrzr-)metadatar new_metadatar]valuesrs ` @r7biom_meta_to_stringrJs$ (C  [111 Hd # #??4+{1K)L)LMM hqk4 ( ( L 7 7????Q???##DIIf$5$5666688L)) )  ----#+----- rXct|}|t}|d|S||ddvr||||S|S)z*Convert a biom table to a contigency tableNr;rr) header_key header_valuemetadata_formatter) load_tablerrdelimited_self)biom_frr md_formattables r7convert_biom_to_tabler#es v  E'  ~~=~))1##%%%U^^^77:::##z1=7@$BB B##%%%rXct|tjtjfr8 t |}n#t tf$rtd|zwxYwt|5} t |}n'#t tf$rtd|zwxYw dddn #1swxYwY|S)a\Load a `Table` from a path Parameters ---------- f : str or file-like object The entity to parse Returns ------- Table Raises ------ IOError If the path does not exist TypeError If the data in the path does not appear to be a BIOM table Examples -------- Parse a table from a path. BIOM will attempt to determine if the fhe file is either in TSV, HDF5, JSON, gzip'd JSON or gzip'd TSV and parse accordingly: >>> from biom import load_table >>> table = load_table('path/to/table.biom') # doctest: +SKIP z%%s does not appear to be a BIOM file!N) rioIOBaseh5pyFilerr/ TypeErrorr)fr"fps r7rrws6:!bi+,, M I$Q''EEI& I I ICaGHH H Iq\\ MR M(,, * M M M G! KLLL M M M M M M M M M M M M M M M M Ls38$A.B40B?B4$B$$B44B8;B82.1.0c .|dkrtd|sdti}t|tjtjfr|j|fi|dSt|d5}|j|fi|ddddS#1swxYwYdS)aUSave a `Table` to a path Parameters ---------- t : biom.Table The table to save f : str or file-like object Where to save format_ : str The format to write. Currently only supports 2.1.0 Returns ------- None Raises ------ IOError If the path is not writable r,z%Only 2.1.0 is supported at this time. generated_bywN)rBrrr'r(Groupto_hdf5r)rr*format_r outs r7 save_tabler4s*'@AAA 1 +--0!di,--% !v q#   %# AIc $ $V $ $ $ % % % % % % % % % % % % % % % % % %s.B  BB)Nr<F)rA)NNN)r,)2numpyrr%r'biom.exceptionrr biom.tabler biom.utilrrr collectionsr __author__ __copyright__ __credits__ __license____url____maintainer__ __email__r r rnMATRIX_ELEMENT_TYPEr+r*r. JSON_SKIPr)r8rWr^rgrirJrKrrrrdictrrr rr#rr4rlrXr7rDsq ????????,,,,,,,, ###### G ,,,  "" * !Ec!EcCC  #J 3Z " " "  "+&+&+&\;7;7;7|*** *** + + + +++"$*$*$*NYOYOYOx[ [ [ [ |s"s"s"s"s"$s"s"s"l*** , , ,6AE$(&&&&$(((V%%%%%%rX