d6mdZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z GddZedkr&d Zejd d Zeejd ejd eZeejdejd eZeejdejd eZeejdejd eZedZeeeZedeee dSdS)zHere is defined the Expr class.N)PerformanceWarning)IO_BUFFER_SIZE BUFFER_TIMESc`eZdZdZiZ ddZddZddZddZdd Z d Z d Z dd Z d Z dZdS)ExprafA class for evaluating expressions with arbitrary array-like objects. Expr is a class for evaluating expressions containing array-like objects. With it, you can evaluate expressions (like "3 * a + 4 * b") that operate on arbitrary large arrays while optimizing the resources required to perform them (basically main memory and CPU cache memory). It is similar to the Numexpr package (see :ref:`[NUMEXPR] `), but in addition to NumPy objects, it also accepts disk-based homogeneous arrays, like the Array, CArray, EArray and Column PyTables objects. .. warning:: Expr class only offers a subset of the Numexpr features due to the complexity of implement some of them when dealing with huge amount of data. All the internal computations are performed via the Numexpr package, so all the broadcast and upcasting rules of Numexpr applies here too. These rules are very similar to the NumPy ones, but with some exceptions due to the particularities of having to deal with potentially very large disk-based arrays. Be sure to read the documentation of the Expr constructor and methods as well as that of Numexpr, if you want to fully grasp these particularities. Parameters ---------- expr : str This specifies the expression to be evaluated, such as "2 * a + 3 * b". uservars : dict This can be used to define the variable names appearing in *expr*. This mapping should consist of identifier-like strings pointing to any `Array`, `CArray`, `EArray`, `Column` or NumPy ndarray instances (or even others which will tried to be converted to ndarrays). When `uservars` is not provided or `None`, the current local and global namespace is sought instead of `uservars`. It is also possible to pass just some of the variables in expression via the `uservars` mapping, and the rest will be retrieved from the current local and global namespaces. kwargs : dict This is meant to pass additional parameters to the Numexpr kernel. This is basically the same as the kwargs argument in Numexpr.evaluate(), and is mainly meant for advanced use. Examples -------- The following shows an example of using Expr:: >>> f = tb.open_file('/tmp/test_expr.h5', 'w') >>> a = f.create_array('/', 'a', np.array([1,2,3])) >>> b = f.create_array('/', 'b', np.array([3,4,5])) >>> c = np.array([4,5,6]) >>> expr = tb.Expr("2 * a + b * c") # initialize the expression >>> expr.eval() # evaluate it array([14, 24, 36], dtype=int64) >>> sum(expr) # use as an iterator 74 where you can see that you can mix different containers in the expression (whenever shapes are consistent). You can also work with multidimensional arrays:: >>> a2 = f.create_array('/', 'a2', np.array([[1,2],[3,4]])) >>> b2 = f.create_array('/', 'b2', np.array([[3,4],[5,6]])) >>> c2 = np.array([4,5]) # This will be broadcasted >>> expr = tb.Expr("2 * a2 + b2-c2") >>> expr.eval() array([[1, 3], [7, 9]], dtype=int64) >>> sum(expr) array([ 8, 12], dtype=int64) >>> f.close() .. rubric:: Expr attributes .. attribute:: append_mode The append mode for user-provided output containers. .. attribute:: maindim Common main dimension for inputs in expression. .. attribute:: names The names of variables in expression (list). .. attribute:: out The user-provided container (if any) for the expression outcome. .. attribute:: o_start The start range selection for the user-provided output. .. attribute:: o_stop The stop range selection for the user-provided output. .. attribute:: o_step The step range selection for the user-provided output. .. attribute:: shape Common shape for the arrays in expression. .. attribute:: values The values of variables in expression (list). Nc d|_ d|_ g|_ d|_ d|_ d|_ d|_ d|_ d\|_|_ |_ d|_ d|_ d|_ g|_ d|_ d|_ |||}tj|}tj||\|_}|D]\}}t)|t*t,t.fvr)t1|t2jt2jfs#t9|dr`t;d|z|jj} | dvrt;d|z|D]O\}}t1|t@j!r0|j"j#s$|j$dkr|%}|||<P|j } g} |jD]}||} t9| dr| &| j'nOt9| dr| &| n)tAj(| } | &| | &| d tS|j| D} tjj*|| fi||_ |+\|_|_dS) NFrNNNdtypezUnsupported variable type: %r)ArrayCArrayEArrayColumnratomcVg|]&\}}|tj|f'S)ne necompilergetType).0nametype_s 1lib/python3.11/site-packages/tables/expression.py z!Expr.__init__..sGCCC&$BM11%889CCC), append_modemaindimnamesouto_starto_stopo_stepshapestartstopstepvalues_compiled_expr_single_row_out_required_expr_varsrr getContext getExprNamesitemstypeintfloatstr isinstancetbLeafrhasattr TypeError __class____name__npndarrayflagsalignedndimcopyappendrarrayzipNumExpr _guess_shape)selfexpruservarskwargsvars_context_rvarobjnamer'types_value signatures r__init__z Expr.__init__s B = :N E D D 8+6( DIty 6 5 5 ;"&#<((x88-**622 224AA  A G GID#CyyS%---cBGRY#788 G3(( ?# EFFFm,GEEE ?# EFFFF  * *ID##rz** *y(*x1}}!hhjj&)d J ! !D$KEuf%% % ej))))(( % e$$$$ e$$$ MM% CC*-dj&*A*ACCC !m3D)NNvNN$(#4#4#6#6  DLLLrcz|j}||vrXt|dkrt|ddD]}||=t|dd}d|jD}|||<n||}ii} }|"t j|} | j}| j} i} |D]} | | |vr || } n0| | |vr || } n!| | | vr | | } ntd| zt| dr*| j j d dd krtd | zt| d rtd | z| | | <| S)aBGet the variables required by the `expression`. A new dictionary defining the variables used in the `expression` is returned. Required variables are first looked up in the `uservars` mapping, then in the set of top-level columns of the table. Unknown variables cause a `NameError` to be raised. When `uservars` is `None`, the local and global namespace where the API callable which uses this method is called is sought instead. To disable this mechanism, just specify a mapping as `uservars`. Nested columns and variables with an ``uint64`` type are not allowed (`TypeError` and `NotImplementedError` are raised, respectively). `depth` specifies the depth of the frame in order to reach local or global variables. N zevalc@g|]}|dvr|tjjv|S))NoneFalseTrue)r expressions functions)rrKs rrz,Expr._required_expr_vars..s@AAA&???r~'??????rzname ``%s`` is not definedr ru8zmvariable ``%s`` refers to a 64-bit unsigned integer object, that is not yet supported in expressions, sorry; _v_colpathnameszEvariable ``%s`` refers to a nested column, not allowed in expressions)_exprvars_cachelenlistcompileco_namessys _getframef_locals f_globals NameErrorr5r r1NotImplementedErrorr6)rD expressionrFdepthexprvars_cachekcexprexprvars user_locals user_globals user_framereqvarsrKvals rr*zExpr._required_expr_varss.- ^ + +>""S((n--crc2**A&q))J F;;EAAu~AAAH*2N: & &%j1H%'\  u--J$-K%/L  C#xsm!c[&8&8!#&!c\&9&9"3' .s900061cCI$.....r)rr$r%r&indicesrr_rangerr9emptyr)r r5rr`r#r r!r"popryr enumerater'r)rDr#ritermoder$r%r&i_nrows o_maindimrr r!r"o_shapes tr_oshapeolen_tr_shapelen_ slice_posrrrsnrowsr=s @r _get_infozExpr._get_infos  "' DIty#2#229'%.2I2I UD$ gE%t$<$< = =??E'NGnGG * E4G' )IxhuD,@,FGGG&12E'NAffWW09-Wfffh3 **, # I sy//$, T[AA*+))GI4F*G*G'%();),U7FF-K-K)L)L&N&N "!MM ! i00;;&#<< 22DDD(($KMMM')EDLL',E)$ 5=D5zz0000Yt{%;%;000   ,, ' 'FAsI~~--c22:%%!&J GYtT:GVV= =YtT:F Frc |j|j|j}}}|||\ }}}}}} } } } } }|dkr||kr |j| S|jSt dg|dzz}t dg| dzz}|D]}t|drd|_t|||| zD]}||| zz}||kr|}t |||||<g}t|D]T\}}||vr6| | t|?| |U|j|}|jr| || ||z |zz}|| |zz}|| kr| }t ||||| <|| t|<|D]}t|drd|_| S)aEvaluate the expression and return the outcome. Because of performance reasons, the computation order tries to go along the common main dimension of all inputs. If not such a common main dimension is found, the iteration will go along the leading dimension instead. For non-consistent shapes in inputs (i.e. shapes having a different number of dimensions), the regular NumPy broadcast rules applies. There is one exception to this rule though: when the dimensions orthogonal to the main dimension of the expression are consistent, but the main dimension itself differs among the inputs, then the shortest one is chosen for doing the computations. This is so because trying to expand very large on-disk arrays could be too expensive or simply not possible. Also, the regular Numexpr casting rules (which are similar to those of NumPy, although you should check the Numexpr manual for the exceptions) are applied to determine the output type. Finally, if the setOuput() method specifying a user container has already been called, the output is sent to this user-provided container. If not, a fresh NumPy container is returned instead. .. warning:: When dealing with large on-disk inputs, failing to specify an on-disk container may consume all your available memory. rNrrFT)r'r#rrr$r)rr5 _v_convertrrr?rtupler(r)rDr'r#rrrr$r%r&rrrr r!r"i_sliceso_slicesrsstart2stop2rrroutstart3stop3s rrUz Expr.evals?@"&dj$,w NN5' * * 2)UD$ i&& a<<}}!7 ++$KK=GaK0$KK=IM2 ' 'CsI&& '!&E4 ):;; , ,FTJ..Et|| %feT : :HW D#F++ % %3 >>KKh @ @AAAAKK$$$$&4&-D , 4    !FUNt#;;f!446>>"E&+FE6&B&B#'+E(OO$$ & &CsI&& &!% rc #K|j|j|j}}}|||d\}}}}}} |dkrdSt dg|dzz} |D]} t | drd| _t|||| zD]} | || zz} | |kr|} t | | || |<g}t|D]T\}} ||vr6| | t| ?| | U|j |}|Ed{V|D]} t | drd| _dS)zIterate over the rows of the outcome of the expression. This iterator always returns rows as NumPy objects, so a possible out container specified in :meth:`Expr.set_output` method is ignored here. T)rrNrrF) r'r#rrrr5rrrr?rrr()rDr'r#rrrr$r%r&rrrsrrrrrs r__iter__z Expr.__iter__ss"&dj$,w NN5'DN 9 9 <)UD$  a<< F$KK=GaK0 ' 'CsI&& '!&E4 ):;;  FTJ..Et|| %feT : :HW D#F++ % %3 >>KKh @ @AAAAKK$$$$&4&-DOOOOOOOO & &CsI&& &!% & &r)N)rQr )F)r8 __module__ __qualname____doc__r^rPr*rvrzr}rrCrrUrrrrrrsppdO X7X7X7X7xKKKKZ ''''@&B...`MGMGMGMG^]]]~4&4&4&4&4&rr__main__)rTi'z/tmp/expression.h5wa)dflt)rr#brQcrz a * b + cz returned-->)!rrcrnumexprrnumpyr9tablesr3 exceptionsr parametersrrrr8r# open_filef create_carrayroot Float32AtomrrrrrErzrUdprintreprcloserrrrs%% ******44444444Y &Y &Y &Y &Y &Y &Y &Y &x z E )3//A .".a*@*@*@NNA .".a*@*@*@NNA .".a*@*@*@NNA //!&%nbn!.D.D.D %  ' 'C 4  DOOC A E-a!!!GGIIIII+r