Cd& ddlmZddlZddlZddlZddlmZmZddl m Z ddl m Z ddlmZdddddddddd Z d Zdd lmZdS) ) annotationsN)baseutils) blockwise)unpack_collections)HighLevelGraphT)nametokendtype adjust_chunksnew_axes align_arrays concatenatemetac   |} |pi}tt|t|kr>tddtj|Dt|d| dddDz t|pdz } | rtd| d d lm}m}|r || \}n~d tj d| D}i|D]O\} t|j D]4\}}|vs&t|t|kr||<5P| ddd}|D]"\}}t|ts|f}||<#t|| ddd}i}g}g}g}|D]\} 3||}t|\}}||n|t!|d rAt! d r1|jt krtd |jfz|j||j<|||j}|| fi}| D]<\}}||}t|\}}|||||<=| sPd|p&t-j|dt3j||||fi| } t7|| |g|R||||d|}t9j| |||z}fd|D}rZt=|D]I\} vr>t? r-ttA ||||<Mt tBj"r&t fd||D||<t ttFfrt t||kr?td|dt||dt dt ||<:tIdKt|}| d dl%m&}|||g| dddRi| } tO|| || |S)a<Tensor operation: Generalized inner and outer products A broad class of blocked algorithms and patterns can be specified with a concise multi-index notation. The ``blockwise`` function applies an in-memory function across multiple blocks of multiple inputs in a variety of ways. Many dask.array operations are special cases of blockwise including elementwise, broadcasting, reductions, tensordot, and transpose. Parameters ---------- func : callable Function to apply to individual tuples of blocks out_ind : iterable Block pattern of the output, something like 'ijk' or (1, 2, 3) *args : sequence of Array, index pairs You may also pass literal arguments, accompanied by None index e.g. (x, 'ij', y, 'jk', z, 'i', some_literal, None) **kwargs : dict Extra keyword arguments to pass to function dtype : np.dtype Datatype of resulting array. concatenate : bool, keyword only If true concatenate arrays along dummy indices, else provide lists adjust_chunks : dict Dictionary mapping index to function to be applied to chunk sizes new_axes : dict, keyword only New indexes and their dimension lengths align_arrays: bool Whether or not to align chunks along equally sized dimensions when multiple arrays are provided. This allows for larger chunks in some arrays to be broken into smaller ones that match chunk sizes in other arrays such that they are compatible for block function mapping. If this is false, then an error will be thrown if arrays do not already have the same number of blocks in each dimension. Examples -------- 2D embarrassingly parallel operation from two arrays, x, and y. >>> import operator, numpy as np, dask.array as da >>> x = da.from_array([[1, 2], ... [3, 4]], chunks=(1, 2)) >>> y = da.from_array([[10, 20], ... [0, 0]]) >>> z = blockwise(operator.add, 'ij', x, 'ij', y, 'ij', dtype='f8') >>> z.compute() array([[11, 22], [ 3, 4]]) Outer product multiplying a by b, two 1-d vectors >>> a = da.from_array([0, 1, 2], chunks=1) >>> b = da.from_array([10, 50, 100], chunks=1) >>> z = blockwise(np.outer, 'ij', a, 'i', b, 'j', dtype='f8') >>> z.compute() array([[ 0, 0, 0], [ 10, 50, 100], [ 20, 100, 200]]) z = x.T >>> z = blockwise(np.transpose, 'ji', x, 'ij', dtype=x.dtype) >>> z.compute() array([[1, 3], [2, 4]]) The transpose case above is illustrative because it does transposition both on each in-memory block by calling ``np.transpose`` and on the order of the blocks themselves, by switching the order of the index ``ij -> ji``. We can compose these same patterns with more variables and more complex in-memory functions z = X + Y.T >>> z = blockwise(lambda x, y: x + y.T, 'ij', x, 'ij', y, 'ji', dtype='f8') >>> z.compute() array([[11, 2], [23, 4]]) Any index, like ``i`` missing from the output index is interpreted as a contraction (note that this differs from Einstein convention; repeated indices do not imply contraction.) In the case of a contraction the passed function should expect an iterable of blocks on any array that holds that index. To receive arrays concatenated along contracted dimensions instead pass ``concatenate=True``. Inner product multiplying a by b, two 1-d vectors >>> def sequence_dot(a_blocks, b_blocks): ... result = 0 ... for a, b in zip(a_blocks, b_blocks): ... result += a.dot(b) ... return result >>> z = blockwise(sequence_dot, '', a, 'i', b, 'i', dtype='f8') >>> z.compute() 250 Add new single-chunk dimensions with the ``new_axes=`` keyword, including the length of the new dimension. New dimensions will always be in a single chunk. >>> def f(a): ... return a[:, None] * np.ones((1, 5)) >>> z = blockwise(f, 'az', a, 'a', new_axes={'z': 5}, dtype=a.dtype) New dimensions can also be multi-chunk by specifying a tuple of chunk sizes. This has limited utility as is (because the chunks are all the same), but the resulting graph can be modified to achieve more useful results (see ``da.map_blocks``). >>> z = blockwise(f, 'az', a, 'a', new_axes={'z': (5, 5)}, dtype=x.dtype) >>> z.chunks ((1, 1, 1), (5, 5)) If the applied function changes the size of each chunk you can specify this with a ``adjust_chunks={...}`` dictionary holding a function for each index that modifies the dimension size in that index. >>> def double(x): ... return np.concatenate([x, x]) >>> y = blockwise(double, 'ij', x, 'ij', ... adjust_chunks={'i': lambda n: 2 * n}, dtype=x.dtype) >>> y.chunks ((2, 2), (2,)) Include literals by indexing with None >>> z = blockwise(operator.add, 'ij', x, 'ij', 1234, None, dtype=x.dtype) >>> z.compute() array([[1235, 1236], [1237, 1238]]) z-Repeated elements not allowed in output indexc$g|] \}}|dk |S)).0kvs 4lib/python3.11/site-packages/dask/array/blockwise.py zblockwise..s! G G G41aQQc h|] }||D]}| SNr)rargas r zblockwise..s# C C Ccosoo!1oooorrNrzUnknown dimensionr) normalize_arg unify_chunksc g|] \}}|||f Srr)rris rrzblockwise..s!RRRfq!AMAq6MMMrndim__len__z1Index string %s does not match array dimension %dz{}-{}_) numblocks dependenciesr r)r)c g|] }| Srr)rr$chunksss rrzblockwise..s * * *Qgaj * * *rc3(K|] }V dSrr)rr'r inds r zblockwise.. s(%L%LQmC&8%L%L%L%L%L%Lrz Dimension z has z& blocks, adjust_chunks specified with z blocksz4adjust_chunks values must be callable, int, or tuple) compute_meta)rr )(lenset ValueErrortoolz frequenciesitemsdask.array.corer!r" partitionzipchunks isinstancetuplerextendhasattrr%r(r appendformatrfuncnamestriprtokenizecore_blockwiserfrom_collections enumeratecallablemapnumbersIntegrallistNotImplementedErrordask.array.utilsr/ new_da_object)!funcout_indr r r r r rrrargskwargsoutnewr!r"arraysargindsrcr$rrr(r) argindsstr collectionskwargs2graphr9r/r+r-s! ` @@rrrs4l C~2H 3w<<CLL(( ; G G5,W55;;== G G G   G C CQTT C C C D hn"     3,c222;;;;;;;; &,-RR4(@(@RRR   # #HCCJ,, # #1G##s1vvGAJ'?'?!"GAJ #ccc  1!U## A &$qt!t*%%GIL FJ&&S ;-$$C1#66 C    , , , ,V$$ C++ HC(( GCHo&#&-Ich  MM#   (C3*%%%%G 1 M!  +A..;K(((   nn  4U^D))//44 M$U E Ef E E          !     E  + U,!6   E+ * * *' * * *F((  FAsm##M#.// %c-*$sss)>>>v>> V$e D D DDrcBtjdt|i|S)Nz.The da.atop function has moved to da.blockwise)warningswarnr)rPrQs ratopr^"s& MBCCC d %f % %%r)rM) __future__rrHr\tlzr3daskrrdask.blockwiserrC dask.delayedrdask.highlevelgraphrr^r6rMrrrres""""""666666++++++......     QEQEQEQEQEh&&& *)))))))r