d%dZddlZddlZddlZddlmZddlmZddl m Z m Z m Z ddl mZddlmZmZmZmZmZmZd ZGd d ejeZGd d eZdS)z Here is defined the Array class.N) hdf5extension)Filters) flavor_ofarray_as_internalinternal_to_flavor)Leaf)is_idxconvert_to_np_atom2SizeTypelazyattr byteordersquantizez2.4ceZdZdZdZedZedZedZ edZ d!fd Z d Z d Z d Zd"dZdZdZdZdZdZdZdZdZdZdZdZdZdZdZd#dZd$dZdZ d Z!xZ"S)%Arraya This class represents homogeneous datasets in an HDF5 file. This class provides methods to write or read data to or from array objects in the file. This class does not allow you neither to enlarge nor compress the datasets on disk; use the EArray class (see :ref:`EArrayClassDescr`) if you want enlargeable dataset support or compression features, or CArray (see :ref:`CArrayClassDescr`) if you just want compression. An interesting property of the Array class is that it remembers the *flavor* of the object that has been saved so that if you saved, for example, a list, you will get a list during readings afterwards; if you saved a NumPy array, you will get a NumPy object, and so forth. Note that this class inherits all the public attributes and methods that Leaf (see :ref:`LeafClassDescr`) already provides. However, as Array instances have no internal I/O buffers, it is not necessary to use the flush() method they inherit from Leaf in order to save their internal state to disk. When a writing method call returns, all the data is already on disk. Parameters ---------- parentnode The parent :class:`Group` object. .. versionchanged:: 3.0 Renamed from *parentNode* to *parentnode* name : str The name of this node in its parent group. obj The array or scalar to be saved. Accepted types are NumPy arrays and scalars as well as native Python sequences and scalars, provided that values are regular (i.e. they are not like ``[[1,2],2]``) and homogeneous (i.e. all the elements are of the same type). .. versionchanged:: 3.0 Renamed form *object* into *obj*. title A description for this node (it sets the ``TITLE`` HDF5 attribute on disk). byteorder The byteorder of the data *on disk*, specified as 'little' or 'big'. If this is not specified, the byteorder is that of the given `object`. track_times Whether time data associated with the leaf are recorded (object access time, raw data modification time, metadata change time, object birth time); default True. Semantics of these times depend on their implementation in the HDF5 library: refer to documentation of the H5O_info_t data structure. As of HDF5 1.8.15, only ctime (metadata change time) is implemented. .. versionadded:: 3.4.3 ARRAYc|jjS)z9The NumPy ``dtype`` that most closely matches this array.)atomdtypeselfs ,lib/python3.11/site-packages/tables/array.pyrz Array.dtypeUsycZ|jdkrtdS|j|jS)z The number of rows in the array.r)shaper maindimrs rnrowsz Array.nrowsZs, :  A;; :dl+ +rcv|j}|jj}t|jD]\}}||kr||z}|S)zLThe size of the rows in bytes in dimensions orthogonal to *maindim*.)rrsize enumerater)rrrowsizeidims rr"z Array.rowsizebsI,). ++  FAsG||3rc |j|jzS)zSThe size of this array's data in bytes when it is fully loaded into memory.)rr"rs rsize_in_memoryzArray.size_in_memorymszDL((rNTc d|_ |dux|_} ||_ ||_ d|_ d|_ d|_ d|_ d|_ d|_ d|_ d|_ d|_ d|_ ||_ d|_ d|_ d|_ t%||| t)|||dS)NTF) _v_version_v_new _v_new_title_obj _v_convert_start_stop_step _nrowsread_startb_stopb_row_initlistarrrrnrowextdimsuper__init__r) r parentnodenameobjtitle byteorder_log_atom track_timesnew __class__s rr;zArray.__init__ss  /tO+ c?!&  D 5 5 2G . / 2 M *   , A 5 T3 9d$ & & & & &rcTt|_ t|jx|_}t |j|}n%#t $r|dwxYw|jj dvrtdd|_| ||jj } | ||j|j\|_|_|_n%#t $r|dwxYw||_d|_|jS)zSave a new array in file.r)flush)VUOzGArray objects cannot currently deal with void, unicode or object arraysN) obversionr*rr-_flavorr Exceptioncloserkind TypeError_g_fix_byteorder_datar@ _create_arrayr,r _v_objectidr_calc_nrowsinbuf nrowsinbuf _v_chunkshape)rflavornparrs r _g_createzArray._g_createsI$ $-di$8$8 8DL6%di88EE    JJQJ      ;  . .788 8 **5%+2GHH 9=8J8Jt($)9595 5T tz499    JJQJ      //11!s0?"A!)4C"Dc|\}|_|_|_||_|S)z+Get the metadata info for an array in file.) _open_arrayrrrVrTrU)roids r_g_openz Array._g_opens><@;K;K;M;M8diT%7//11 rch|jjdkrtd|jz|jjS)aGet the enumerated type associated with this array. If this array is of an enumerated type, the corresponding Enum instance (see :ref:`EnumClassDescr`) is returned. If it is not of an enumerated type, a TypeError is raised. enumz)array ``%s`` is not of an enumerated type)rrOrP _v_pathnamer_rs rget_enumzArray.get_enums? 9>V # #G"./00 0y~rc ||||\|_|_|_n#t$rYdSwxYw||S)axIterate over the rows of the array. This method returns an iterator yielding an object of the current flavor for each selected row in the array. The returned rows are taken from the *main dimension*. If a range is not supplied, *all the rows* in the array are iterated upon - you can also use the :meth:`Array.__iter__` special method for that purpose. If you only want to iterate over a given *range of rows* in the array, you may use the start, stop and step parameters. Examples -------- :: result = [row for row in arrayInstance.iterrows(step=4)] .. versionchanged:: 3.0 If the *start* parameter is provided and *stop* is None then the array is iterated from *start* to the last line. In PyTables < 3.0 only one element was returned. r)_process_ranger/r0r1 IndexError _init_loop)rstartstopsteps riterrowszArray.iterrowssk4 484G4GtT5#5# 1T[$*djj   22   s *- ;;cp|js.d|_|j|_d|_||S)azIterate over the rows of the array. This is equivalent to calling :meth:`Array.iterrows` with default arguments, i.e. it iterates over *all the rows* in the array. Examples -------- :: result = [row[2] for row in array] Which is equivalent to:: result = [row[2] for row in array.iterrows()] rr)r6r/rr0r1rers r__iter__zArray.__iter__s:&z DKDJDJ OO    rc|j|_|j|_d|_d|_t |j|jz |_dS)z)Initialization for the __iter__ iterator.r)TN)r/r2r3r5r6r r1r8rs rrezArray._init_loop1s@+{   T[4:566 rc|j|jkrd|_d|_t|jdz|jks |jdkr|j|j|jzz|_ |j |jkr |j|_ | |j|j |j}|j dkr| |j d}t||j|_d|_|j |_|xjdz c_|xj|jz c_|xj|jz c_|jr|j|jS|jS)zGet the next element of the array during an iteration. The element is returned as an object of the current flavor. FNrrr))r2r0r6r7 StopIterationr5rUr3r1r4_readr9swapaxesrrWr8r)rr7s r__next__zArray.__next__:s+ ?dj ( (DJDL y1}//49q=="lTZ$/-II ;++"&*DK**T\4; KK;??%..t{A>>G1'4;GG  #{ IINII II #II OOtz )OOz $|DI..|#rc t|j}|f}tj|t}tj|t}tj|t}tj|t}t |ts|f}t|}d} |D]}} d} t | ttrPd} t| t|j|| z z dzD]!} d|| <|j| || <d|| <| dz } "n| |krtd|j zt| rptj| } | |j| krtd| dkr| |j| z } || | dzd| \} }}d|| <nTt | t"r-|| j| j| j| \} }}nt+d| z| s| || <||| <||| <| dz } | t|jkrDt| t|jD]!} d|| <|j| || <d|| <| dz } "g}tt|jD]U} tt|| || || }|dkr|| s||V||||fS)z4Internal routine used by __getitem__ and __setitem__rrrrz Too many indices for object '%s'zIndex out of range)r$zNon-valid index or slice: %s)lenrnpemptyr zeros isinstancetupletypeEllipsisrangerdr`r operatorindexrcslicerfrgrhrPappend)rkeysmaxlenrstartlstoplstepl stop_Nonenkeysr$keyellipsisdimlrfrgrhnew_dims r_interpret_indexingzArray._interpret_indexing_sLTZ X666uH555uH555H5999 $&& 7DD ! ! CH#tH~~.. F!#s4:%#+'F'JKKD"#F3K!%D!1E#J!"E#J1HCC   !C!%!1"2333 FnS))$*S/))$%9:::774:c?*C$($7$7q!%8%.%.!tT!" #C'' F$($7$7Isxs%8%<%<!tTT > DEEE #s !c !c q TZ c3tz??33  s !Z-c c qTZ)) & &C%s U3ZsDDEEGqLLYs^L W%%%ueU**rc d}d}d}g}g}t|ts|f}||t|j}d}d}t t ||jD]`\} \} } t| t r@|| | \} } }|| | || df|| ^ t| } |t| n(#t$r| g} |dYnwxYwt| dkrtd| zt| d kr|rtd d }nt| dttj frJt| dtjr9tj| djtj std tj| d }tj|dk| |z|}t|ttj|krtd|}|jt| fksLtjtj|tjt| z dkr9|tdt+d|Dd z }||f}||}t1t|d zD]}|t|kr|||| |dkr d} |d} n=|t|kr|dd z} | | z } n||d z d z} ||| z } | dkr|| | d | dfbtd|D}|||fS)a3Performs a NumPy-style fancy selection in `self`. Implements advanced NumPy-style selection operations in addition to the standard slice-and-int behavior. Indexing arguments may be ints, slices or lists of indices. Note: This is a backport from the h5py project. c t|}n #t$rtd|zwxYw||dz krtd|zdS)z1Validate a list member for the given axis length.zIllegal index: %rrzIndex out of bounds: %dN)intrPrd)numlengths rvalidate_numberz/Array._fancy_selection..validate_numbersn ;#hh ; ; ; 3c 9::: ;VaZ !:S!@AAA s/ctd|D}|dkrtd|dkrt||kr |tfz}g}t|}t |D]P\}}|tur-|t df||z dzz;||Qt||krtd|S)z1Expand ellipsis objects and fill in missing axes.c3,K|]}|tu dVdS)rN)r{).0args r zBArray._fancy_selection..expand_ellipsis..s&<.expand_ellipsiss<= 1 (got %d)z&Zero-length selections are not allowedz(Reverse-order selections are not allowedz"Start index %s out of range (0-%d)z!Stop index %s out of range (1-%d)z)Selection out of bounds (%d; axis has %d))rfrgrhrrd)exprrfrgrhcounts rtranslate_slicez/Array._fancy_selection..translate_slices!$ 38SX4E}E |4yy|4yyaxx !=!DEEEu}} !IJJJe|| !KLLLqyyaxx}----&1*---- 8E6A:;NNPPP&&&&&&&& 74.HJJJE\d*Eu $)) u}v%% ?U]F+,---%% %rFNANDrz*Empty selections are not allowed (axis %d)rz"Only one selection list is allowedTz!Only integer coordinates allowed.i8rz+Selection lists cannot have repeated valuesz(Only one selection list can be unorderedc3&K|] }|dkdV dS)rrNrrxs rrz)Array._fancy_selection..6s&'D'DaQ!VVVVVV'D'Drr)NOTBc3&K|] }|dk|V dS)rNrrs rrz)Array._fancy_selection..Ns&33QAFFqFFFF33r)rxryrtrr!ziprrlistrPrdrruintegerndarray issubdtyperasarraywhereuniqueargsortrabsaranger|)rrrrrmshape selection list_seenreorderrrrrfrrhnexpneworder corrected_idx select_idxs r_fancy_selectionzArray._fancy_selectionsN B B B   ,0 &0 &0 &f $&& 7DtS__55 "+Cdj,A,A"B"B? I? I C#v#u%%> I%4_S&%A%A"ud  %c5!ABBB e$$$$,s))C MM#c((++++ !%%%%CMM!$$$$$% s88q==$DsJLLLXX\\ )()MNNN$( &s1vRZ/@AAM#CFBJ77MM#a&, CCM((KLLLz#T222xq&4->>t99BIdOO 4 444$EGGG<<>>Ns3xxk11rvh3s881D1D&DEEFF!KK*(FHHH$''D'D6'D'D'D$D$Dq$HM,h7G>D"'D A "6"6IIJ"CII--'Z(8&AAA!Q ! $Q#s4yy00 $R1  & $Z!^ 4q 8 $Z 05 8qyy!((%3)GHHH'I*33&33333'6))s C;;"D D c| ||\}}}}|||||}n|#t$ro ||}||}n@#t$r3||\}} }||| |}YnwxYwYnwxYw|jdks|j s|St||jS)acGet a row, a range of rows or a slice from the array. The set of tokens allowed for the key is the same as that for extended slicing in Python (including the Ellipsis or ... token). The result is an object of the current flavor; its shape depends on the kind of slice used as key and the shape of the array itself. Furthermore, NumPy-style fancy indexing, where a list of indices in a certain axis is specified, is also supported. Note that only one list per selection is supported right now. Finally, NumPy-style point and boolean selections are supported as well. Examples -------- :: array1 = array[4] # simple selection array2 = array[4:1000:2] # slice selection array3 = array[1, ..., ::2, 1:4, 4:] # general slice selection array4 = array[1, [1,5,10], ..., -1] # fancy selection array5 = array[np.where(array[:] > 4)] # point selection array6 = array[array[:] > 4] # boolean selection numpy) _g_check_openr _read_slicerP_point_selection _read_coordsr_read_selectionrWr.r) rrrrrrarrcoordsrrs r __getitem__zArray.__getitem__Qs#6  F*.*B*B3*G*G 'FE5%""65%??CC F F F F..s33''// F F F,0,A,A#,F,F) 7E**9guEE F  F ;' ! ! !J!#t{333s52A C*A?>C?:B<9C;B<<CCcn|t||j}|jdkrdS|jj>t j|jt j st||jj} | |\}}}}| |||||dS#t$rs ||}|||YdS#t$r6||\} } }|| | ||YYdSwxYwwxYw)aBSet a row, a range of rows or a slice in the array. It takes different actions depending on the type of the key parameter: if it is an integer, the corresponding array row is set to value (the value is broadcast when needed). If key is a slice, the row slice determined by it is set to value (as usual, if the slice to be updated exceeds the actual shape of the array, only the values in the existing range are updated). If value is a multidimensional object, then its shape must be compatible with the shape determined by key, otherwise, a ValueError will be raised. Furthermore, NumPy-style fancy indexing, where a list of indices in a certain axis is specified, is also supported. Note that only one list per selection is supported right now. Finally, NumPy-style point and boolean selections are supported as well. Examples -------- :: a1[0] = 333 # assign an integer to a Integer Array row a2[0] = 'b' # assign a string to a string Array row a3[1:4] = 5 # broadcast 5 to slice 1:4 a4[1:4:2] = 'xXx' # broadcast 'xXx' to slice 1:4:2 # General slice update (a5.shape = (4,3,2,8,5,10). a5[1, ..., ::2, 1:4, 4:] = numpy.arange(1728, shape=(4,3,2,4,3,6)) a6[1, [1,5,10], ..., -1] = arr # fancy selection a7[np.where(a6[:] > 4)] = 4 # point selection + broadcast a8[arr > 4] = arr2 # boolean selection rN)rr rr filtersleast_significant_digitrurr signedintegerrr _write_slicerPr _write_coordsr_write_selection) rrvaluerXrrrrrrrs r __setitem__zArray.__setitem__snJ $E4955 :?? F L 0 <M%+r/?@@ =UDL$HIIE H*.*B*B3*G*G 'FE5%   feUE5 A A A A A H H H H..s33""65111111 H H H,0,A,A#,F,F) 7E%%i%GGGGGGG H  Hs*3B77 D4+C00;D0+D4/D00D4c|j||jjjzkr'tj||jj}||d<|S|S)zTest that nparr shape is consistent with underlying object. If not, try creating a new nparr object, using broadcasting if necessary. rs.)rrrrurv)rrX slice_shapenarrs r _check_shapezArray._check_shapesJ ;;)>> ? ?8+TY_EEEDDIKLrctj|jj|}d|vr||||||jdkr|d}|S)z4Read a slice based on `startl`, `stopl` and `stepl`.rrrr)rurvrr _g_read_slicer)rrrrrrXs rrzArray._read_slicesZtye<<< E>>   vueU ; ; ; ;"  "IE rctj|jjt |}t |dkr||||jdkr|d}|S)z)Read a set of points defined by `coords`.rrr)rurvrrrt_g_read_coordsrrrrXs rrzArray._read_coordssbtyc&kkBBB v;;??    . . . ;"  "IE rcdtj|jj|}||||jdkr |d}ne|c|\}}t dgt|z}|||<|t| }|S)zA%%/14 FE6599999rct|dkr<||t|f}|||dSdS)z?Write `nparr` values in points defined by `coords` coordinates.rN)rtr_g_write_coordsrs rrzArray._write_coords sS v;;??%%ec&kk^<>>CC!\K7N++ "H3:"H"H1?"H"H"HIII9^, B !@AAAC E>>   UD$ 4 4 4 O39./3=@@ LL    rc&||4|jdkr)d|j}t|||||\}}}|||||}t ||jS)arGet data in the array as an object of the current flavor. The start, stop and step parameters can be used to select only a *range of rows* in the array. Their meanings are the same as in the built-in range() Python function, except that negative values of step are not allowed yet. Moreover, if only start is specified, then stop will be set to start + 1. If you do not specify neither start nor stop, then *all the rows* in the array are selected. The out parameter may be used to specify a NumPy array to receive the output data. Note that the array must have the same size as the data selected with the other parameters. Note that the array's datatype is not checked and no type casting is performed, so if it does not match the datatype on disk, the output will not be correct. Also, this parameter is only valid when the array's flavor is set to 'numpy'. Otherwise, a TypeError will be raised. When data is read from disk in NumPy format, the output will be in the current system's byteorder, regardless of how it is stored on disk. The exception is when an output buffer is supplied, in which case the output will be in the byteorder of that output buffer. .. versionchanged:: 3.0 Added the *out* parameter. NrzXOptional 'out' argument may only be supplied if array flavor is 'numpy', currently is {})rrWformatrP_process_range_readror)rrfrgrhrmsgrs rreadz Array.read?s:  ?t{g5589? 9L9L C.. "66udDIIdjjdC00!#t{333rc  ||||\}}}|jr ||||} n|d} t||| || |j} t j|jt |jjz} | | fS)z2Private part of Leaf.copy() for each kind of leaf.r)r?rArBr)rrrrruprodr r )rgroupr=rfrgrhr?r chunkshaperAkwargsrobject_rs r_g_copy_with_statszArray._g_copy_with_statses #66udDIId : uT$'CCr(C tSD"i)))8444ty~E  rc Z|d|jd|jd|jd|jd|j S)z;This provides more metainfo in addition to standard __str__z atom := z maindim := z flavor := z byteorder := z chunkshape := )rrrWr@rrs r__repr__zArray.__repr__{sg'' 9''l''['' ''  '' 'r)Nr'NTNT)NNN)N)NNNN)#__name__ __module__ __qualname____doc__ _c_classidr rpropertyrr"r&r;rYr]rarirkrerqrrrrrrrrrrrrorrr __classcell__)rEs@rrrs/77tJ X,,X,X))X) "$26!:&:&:&:&:&:&x) ) ) V   !!!!F8777#$#$#$JA+A+A+Fm*m*m*^.4.4.4`<H<H<H|&      0:::000222$:$4$4$4$4L!!!,'''''''rrceZdZdZdZdS) ImageArrayzArray containing an image. This class has no additional behaviour or functionality compared to that of an ordinary array. It simply enables the user to open an ``IMAGE`` HDF5 node as a normal `Array` node in PyTables. IMAGEN)rrrrrrrrr r sJJJrr )rr}rrrur'rrrrWrrrleafr utilsr r r r rrrKrr rrrrs@&& DDDDDDDDDD****************  k 'k 'k 'k 'k 'M k 'k 'k '\          r