HR-eO@dZddlZddlZddlZddlmZddlZddlm Z ddl m Z m Z ddl mZddlmZdd lmZmZd Zd Zd ZGd deZGddejZdZGddejZGddejZGddejZ dS)z Define the Enhanced Character-Separated-Values (ECSV) which allows for reading and writing all the meta data associated with an astropy Table object. N) OrderedDict) convert_numpy)meta serialize)serialize_context_as)AstropyUserWarning)basiccorez1.0) ,)boolint8int16int32int64uint8uint16uint32uint64float16float32float64float128stringceZdZdZdS)InvalidEcsvDatatypeWarningz. ECSV specific Astropy warning class. N)__name__ __module__ __qualname____doc__5lib/python3.11/site-packages/astropy/io/ascii/ecsv.pyrr)sr#rc:eZdZdZdZdZdZdZfdZxZ S) EcsvHeaderzHeader class for which the column definition line starts with the comment character. See the :class:`CommentedHeader` class for an example. c#Ktj|j}|D]U}|}|s||}|r#||d}|r|VSdSdS)zReturn only non-blank lines that start with the comment regexp. For these lines strip out the matching characters and leading/trailing whitespace. N)recompilecommentstripmatchend)selflines re_commentliner,outs r$ process_lineszEcsvHeader.process_lines4sZ --   D::<z$EcsvHeader.write..ks!NNNDd(4/NNNr#c&g|]}|jjSr")infonamer;xs r$r<z$EcsvHeader.write..ls(H(H(H(H(H(Hr#N) splitterr8 DELIMITERS ValueErrorr6 table_meta ECSV_VERSIONrget_yaml_from_headerextendappendjoin)r.r/headerheader_yaml_liness` r$writezEcsvHeader.writeFs" = "* 4 4O  )}== ? -!_F6N = "c ) )"&-"9F;  $\ # #    %f - -.  NNNN \d+) \. (?P \d+) \.? (?P \d+)? $zeECSV header line like "# %ECSV " not found as first line. This is required for a ECSV file.rz#unable to parse yaml in meta headerrr8r5c3*K|]}|d|fVdS)r?Nr"r@s r$ z&EcsvHeader.get_cols..s*!M!MQ1V9a.!M!M!M!M!M!Mr#datatypecg|] }|d S)r?r"r@s r$r<z'EcsvHeader.get_cols..s<<>mmC((%dsdm  Jwstt}55 - . .3%%11Cu---{f$$& C! '! 's B$$$C) rrr r!r3rMrPrSr __classcell__rs@r$r&r&/s$&K&K&KP      j'j'j'j'j'j'j'j'j'r#r&cL|jdkrtd|jddS)Nrazdatatype of column z must be "string")r`rDr?)r|s r$_check_dtype_is_strrs3 yELsxLLLMMMr#c,eZdZdZgZfdZdZxZS) EcsvOutputtera After reading the input lines and processing, convert the Reader columns and metadata to an astropy.table.Table object. This overrides the default converters to be an empty list because there is no "guessing" of the conversion function. ctt||}tj|}|SN)rl__call__r_construct_mixins_from_columns)r.r6rr2rs r$rzEcsvOutputter.__call__s3ggtT**6s;; r#c 8 |D]} |jdkrbt|d|jD}tjt |g|jzt|_||jd<n|jr|jdt|t|dr,tj |j dD] }d |j|< g}|jD]}tj |} tj||j}n#t$rtj|t}|dk} tj|jj} d d d | d|| <tj||j| }YnwxYw||d|_tjt|_tjt |t|_||jdd<n|jrst|t|drutj|jdt} tj| } tj |j dD] }| |j|< |` d|jD}tj|t}|dk} tj| s ||j|_ntj|jj} d d d | d|| <tj||j| |_ng|jr4t5jd|jd|jd|jdt:t=|j\} }| |j|_|jjddt?|jkrtAdE#tj!$rtAd|jdtD$r}tAd|jd|d}~wwxYwdS)aREAD: Convert str_vals in `cols` to final arrays with correct dtypes. This is adapted from ``BaseOutputter._convert_vals``. In the case of ECSV there is no guessing and all types are known in advance. A big change is handling the possibility of JSON-encoded values, both unstructured object data and structured values that may contain masked data. rcc6g|]}tj|Sr"rbrsr;vals r$r<z/EcsvOutputter._convert_vals.. HHHC 3HHHr#)r`.Nmaskrz[]r#)US)rr")rt fill_valuer`c6g|]}tj|Sr"rrs r$r<z/EcsvOutputter._convert_vals..Prr#zunexpected subtype z set for column z, using dtype=z instead.r]r z1shape mismatch between value and column specifierzcolumn z2 failed to convert: column value is not valid JSONz failed to convert: )#r\rstr_valsnpemptylenrtrcrihasattrnonzerorrbrsarray TypeErrorr`kindgetmaastyperIfulldumpstolistanyrprqr?rrtuplerDJSONDecodeError Exception)r.r6r|col_valsrstr_valobj_valarr_valrirr all_none_arr all_none_jsonconverter_func_excs r$ _convert_valszEcsvOutputter._convert_valssh Rh RCg R;(**',,,HH3<HHHH!xX#)(C6RRRCH$,CHSMM YO<39R=#8',,, sF++5#%:ch#7#7#:55C04CL-- "H#&<11"&*W"5"5 W&(hwck&J&J&JGG( W W W$&8G6#B#B#BD#'4>C0=CL--HHH3<HHHH8HF;;;D4!""%sy)9)999$K: '    5ch555 R R R !P38!P!P3!P!PQQQ ROh Rh Rs>C-Q5DQBF2/Q1F22JQ0R8RR)rrr r!default_convertersrrrrs@r$rrsg     pRpRpRpRpRpRpRr#rc$eZdZfdZdZxZS)EcsvDatact| |jjd}n#tt f$rYdSwxYw|D]4}|jdkr'|j|vr||jddkri|_5dS)a9READ: Set the fill values of the individual cols based on fill_values of BaseData. For ECSV handle the corner case of data that has been serialized using the serialize_method='data_mask' option, which writes the full data and mask directly, AND where that table includes a string column with zero-length string entries ("") which are valid data. Normally the super() method will set col.fill_value=('', '0') to replace blanks with a '0'. But for that corner case subset, instead do not do any filling. __serialized_columns__Nrarz!astropy.table.column.MaskedColumn) rl_set_fill_valuesrKrEAttributeErrorKeyErrorr`r? fill_values)r.r6scsr|rs r$rzEcsvData._set_fill_valuesys   &&& +()ABCC)    FF   % %C U""HOOM+.2UUU"$  % %s7A  A c|jD]tjdksjjjdkrfdnfd fdt tD_n2#t$r%}tdjj d||d}~wwxYwtd r7j dkr,j d D] }d j|< d |jD}|S) aWRITE: convert all values in table to a list of lists of strings. This version considerably simplifies the base method: - No need to set fill values and column formats - No per-item formatting, just use repr() - Use JSON for object-type or multidim values - Only Column or MaskedColumn can end up as cols here. - Only replace masked values with "", not the generalized filling r Oc|} |}n#t$rYnwxYwtj|dS)N)r :) separators)rrrbr)robjr|s r$format_col_itemz*EcsvData.str_vals..format_col_itemsUc(C!jjll):cjAAAAs  --c.t|Sr)ra)rr|s r$rz*EcsvData.str_vals..format_col_itemss3x==(r#c&g|] }|Sr"r")r;rrs r$r<z%EcsvData.str_vals..s#PPP 4 4PPPr#zcould not convert column z to string: Nrrrcg|] }|j Sr")r)r;r|s r$r<z%EcsvData.str_vals..s111s|111r#)r6rrtr>r`rrangerrr?rndimrr)r.rrr2r|rs @@r$rzEcsvData.str_valss[9 + +C39~~!!SX^%8C%?%?BBBBBB))))) PPPPc#hhPPP    R RRSRR sF## +A 8++--a0++C(*CL%%11ty111 s-A33 B"= BB")rrr rrrrs@r$rrxsH%%%%%@(((((((r#rc4eZdZdZdZdZdZeZe Z e Z dZ dZdS)EcsvaECSV (Enhanced Character Separated Values) format table. Th ECSV format allows for specification of key table and column meta-data, in particular the data type and unit. See: https://github.com/astropy/astropy-APEs/blob/main/APE6.rst Examples -------- >>> from astropy.table import Table >>> ecsv_content = '''# %ECSV 0.9 ... # --- ... # datatype: ... # - {name: a, unit: m / s, datatype: int64, format: '%03d'} ... # - {name: b, unit: km, datatype: int64, description: This is column b} ... a b ... 001 2 ... 004 3 ... ''' >>> Table.read(ecsv_content, format='ascii.ecsv') a b m / s km int64 int64 ----- ----- 001 2 004 3 ecsvz Enhanced CSVz.ecsvNc|td5tj|}dddn #1swxYwY|S)a Update table columns in place if mixin columns are present. This is a hook to allow updating the table columns after name filtering but before setting up to write the data. This is currently only used by ECSV and is otherwise just a pass-through. Parameters ---------- table : `astropy.table.Table` Input table for writing Returns ------- table : `astropy.table.Table` Output table for writing rN)rrrepresent_mixins_as_columns)r.tabler2s r$update_table_datazEcsv.update_table_datas|$"& ) ) ? ?7>>C ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? s 155)rrr r! _format_name _description_io_registry_suffixr& header_classr data_classroutputter_classmax_ndimrr"r#r$rrsS>L!L!LJ#OHr#r)!r!rbr(rp collectionsrnumpyrastropy.io.ascii.corer astropy.tablerrastropy.utils.data_inforastropy.utils.exceptionsrrr r rFrCror BasicHeaderr&rTableOutputterr BasicDatarBasicrr"r#r$rs  ######//////))))))))888888777777  $!3 w'w'w'w'w'"w'w'w'tNNN GRGRGRGRGRD'GRGRGRTIIIIIuIIIX>>>>>5;>>>>>r#