Cd *ddlmZddlZddlZddlmZddlmZddlm Z ddl m Z ddl Z ddlmZddlmZmZdd lmZdd lmZdd lmZdd lmZdd lmZmZdZdZdZ dZ!ddZ"dS)) annotationsN)Counter)reduce)product)mul)config)Arraynormalize_chunks)meta_from_array)tokenize)flatten)HighLevelGraph)M parse_bytesa Dask's reshape only supports operations that merge or split existing dimensions evenly. For example: >>> x = da.ones((6, 5, 4), chunks=(3, 2, 2)) >>> x.reshape((3, 2, 5, 4)) # supported, splits 6 into 3 & 2 >>> x.reshape((30, 4)) # supported, merges 6 & 5 into 30 >>> x.reshape((4, 5, 6)) # unsupported, existing dimensions split unevenly To work around this you may call reshape in multiple passes, or (if your data is small enough) call ``compute`` first and handle reshaping in ``numpy`` directly. c  tdDsJtdz }t|dz }dttD}dtt|D}|dks|dkr|||kr!|||<|||<|dz}|dz}@|}||}|dkr d||<|dz}n'|dkr d||<|dz}n||kr|dz } | dkrStt| |dz|kr/| dz} | dkr$tt| |dz|k/tt| |dz|krt t tfdt|Dr\t|dzD] } | || <|tjtt| |z||<nt| dz|dzD]} | f|| <tttt| dz|dz} t| | || <tt| dz|dztfd|| D||<|dz}| dz }n,||kr%|dz } | dkrStt|| |dz|kr/| dz} | dkr$tt|| |dz|k/tt|| |dz|krt t tt|| dz|dz t| ||<t| dz|dzD]} || f|| <t fd ||D|| <| dz }|dz}|dk|dkt|t|fS) Nc3@K|]}t|tVdSN) isinstancetuple.0cs 2lib/python3.11/site-packages/dask/array/reshape.py z"reshape_rechunk..$s,66z!U##666666cg|]}dSrris r z#reshape_rechunk..'s999t999rcg|]}dSrrrs rr!z#reshape_rechunk..(s;;;;;;rrrc3VK|]#}t||kV$dSr)len)rr inchunksinshapes rrz"reshape_rechunk..Ds8EEa3x{##wqz1EEEEEErc3"K|] }|zV dSrr)rrprods rrz"reshape_rechunk..Rs8--!"D1H------rc3"K|] }|zV dSrr)rrcss rrz"reshape_rechunk..fs'+Q+QAG+Q+Q+Q+Q+Q+Qr) allr%rangerrNotImplementedError_not_implemented_messagemathr)map expand_tuplercontract_tuple)r'outshaper&iioiresult_inchunksresult_outchunksdindoutileftr chunk_reductionoleftr+r)s` ` @@rreshape_rechunkr>#s 66X666 6 6666 W B X B99U3w<<%8%8999O;;eCMM&:&:;;; ''R1WW 2;(2, & &"*2,OB #+B< R !GB !GB bk| !88"&OB  !GBB QYY#' R !GBB 4ZZFE vc7526>+BCCdJJ  vc7526>+BCCdJJc7526>233t;;)*BCCCEEEEE599EEEEE rAv55A)1!OA&&'/|diXeBh/0077( $$uqy"q&1177A*1!*OA&&"(c#x BQRF@R7S.T.T"U"U)5huo)W)W&c7519rAv+=#>??',----&5e&<---(( $ !GBBB 4ZZFE1**Xeb1fn-E!F!F!L!L 1**Xeb1fn-E!F!F!L!Lc8EBFN344;;)*BCCCXeai"q&&89::B"0"r"B"BOB 519b1f-- 5 5'/{n ##&++Q+Q+Q+Q_R=P+Q+Q+Q&Q&Q U #B !GB ''R1WWB  ! !5)9#:#: ::rcf|dkr|Sg}|D]t}|}t||z d}|d|zkr=|t||t|z}|d|zk=|r||ut|t|ksJt |S)z >>> expand_tuple((2, 4), 2) (1, 1, 2, 2) >>> expand_tuple((2, 4), 3) (1, 1, 1, 1, 2) >>> expand_tuple((3, 4), 2) (1, 2, 2, 2) >>> expand_tuple((7, 4), 3) (2, 2, 3, 1, 1, 2) r)maxappendintsumr)chunksfactoroutrxparts rr2r2ns{{ C  1v:q!!1t8mm JJs4yy ! ! ! TNA1t8mm   JJqMMM v;;#c(( " " " " ::rct||zdksJg}d}|D]-}||z }||z}||z}||z}|r||.t|S)zReturn simple chunks tuple such that factor divides all elements Examples -------- >>> contract_tuple((2, 2, 8, 4), 4) (4, 8, 4) r)rDrBr)rErFrGresidualchunkdivgoods rr3r3s v;; 1 $ $ $ $ CH vo6>|   JJt    ::rTc4 ddlm}ddlm}t t ||}d|D}t |t |krt |t |z dkrtdt |dkr |jdkr|S||j tt|dz t fd|D}tj t|jr$tdt!|jztt|d|j krtd |j|kr|St#|t |}d t%||z}|jdkrt)t+|} |fd t |zzt.j| |fi} t d |D} t3j|| |g } t7| || |St |j} t |}|s5| |kr/|dt;| |z D}t=|j||j\}}ttd|D|j j!z}|7tEtGj$d}tGj$dd}ntE|}d}||kr|d}tKj&||dn|rtO|tO|z}g}|D]I}||dkr&|(|||xxdzcc<4|(dJtS||||j |}||}tUtW|j,ggd|DR}tUtW|ggd|DR}tUtW|}dt[|||D} t3j|| |g } t7| |||S)abReshape array to new shape Parameters ---------- shape : int or tuple of ints The new shape should be compatible with the original shape. If an integer, then the result will be a 1-D array of that length. One shape dimension can be -1. In this case, the value is inferred from the length of the array and remaining dimensions. merge_chunks : bool, default True Whether to merge chunks using the logic in :meth:`dask.array.rechunk` when communication is necessary given the input array chunking and the output shape. With ``merge_chunks==False``, the input array will be rechunked to a chunksize of 1, which can create very many tasks. limit: int (optional) The maximum block size to target in bytes. If no limit is provided, it defaults to using the ``array.chunk-size`` Dask config value. Notes ----- This is a parallelized version of the ``np.reshape`` function with the following limitations: 1. It assumes that the array is stored in `row-major order`_ 2. It only allows for reshapings that collapse or merge dimensions like ``(1, 2, 3, 4) -> (1, 6, 4)`` or ``(64,) -> (4, 4, 4)`` .. _`row-major order`: https://en.wikipedia.org/wiki/Row-_and_column-major_order When communication is necessary this algorithm depends on the logic within rechunk. It endeavors to keep chunk sizes roughly the same when possible. See :ref:`array-chunks.reshaping` for a discussion the tradeoffs of ``merge_chunks``. See Also -------- dask.array.rechunk numpy.reshape r)PerformanceWarning)sanitize_indexcg|] }|dk| S)r)rss rr!zreshape..s///qBww1wwwrrz&can only specify one unknown dimensionc3,K|]}|dkrn|VdS)rSNr)rrT missing_sizes rrzreshape..s/DDa2ggll1DDDDDDrz_Array chunk size or shape is unknown. shape: %s Possible solution with x.compute_chunk_sizes()z)total size of new array must be unchangedzreshape-)rc3K|]}|fVdSrr)rds rrzreshape..s$++t++++++r) dependencies)metaci|]}|dSr#rrs r zreshape..s777q!777rc,g|]}t|Sr)rArs rr!zreshape..s)D)D)DQ#a&&)D)D)DrNzarray.chunk-sizez array.slicing.split-large-chunksTaReshaping is producing a large chunk. To accept the large chunk and silence this warning, set the option >>> with dask.config.set(**{'array.slicing.split_large_chunks': False}): ... array.reshape(shape) To avoid creating the large chunks, set the option >>> with dask.config.set(**{'array.slicing.split_large_chunks': True}): ... array.reshape(shape)Explictly passing ``limit`` to ``reshape`` will also silence this warning >>> array.reshape(shape, limit='128 MiB')) stacklevelauto)shapelimitdtypeprevious_chunkscFg|]}tt|Srr-r%rs rr!zreshape..(s$'H'H'H!c!ff 'H'H'HrcFg|]}tt|Srrfrs rr!zreshape..)s$%G%G%GeCFFmm%G%G%Grc6i|]\}}}|tj||fSr)rreshape)rabras rr\zreshape..+s) V V V 1e1qy!U# V V Vr).dask.array.corerPdask.array.slicingrQrr1r% ValueErrorndimsizerrnpisnanrDrastrr r npartitionsnextr __dask_keys__rrirfrom_collectionsr rechunkr-r>rErcitemsizerrgetwarningswarnrrBr listrnamezip)rHra merge_chunksrbrPrQ known_sizesrZr~keydskrEgraphr9r:r& outchunksmax_chunksize_in_bytessplitmsgmatching_chunks chunk_planrGx2in_keysout_keysshapesrVs @rririsT322222111111 #ne,, - -E//e///K ;#e**$$ u::K(( (1 , ,EFF F u::??qv{{H%~afvc;/J/J&JKK DDDDeDDDDD xAG   =?B17|| L   c5!&&DEEEw% 1c%jj ) )D E** *D}71??,,--..wE **QYU,CD++U+++++/cLLLUD&t4444 ag,,C u::D 9C$JJ II77U3:%6%6777 8 8)!'5!(CCHi#C)D)D))D)D)DEEHXX }FJ'9::;; =tDDE""%% =@  M#1a @ @ @ @ @  %h//')2D2DDOJ  . ."3'!++%%c***#C(((A-((((%%f----(g ( I 8  B7BG9I'H'Hx'H'H'HIIIJJGGTFH%G%GY%G%G%GHHHIIH '9% & &F V Vs8Wf7U7U V V VC  +D#RD I I IE id 3 3 33r)TN)# __future__rr0r{ collectionsr functoolsr itertoolsroperatorrnumpyrqdaskrrlr r dask.array.utilsr dask.baser dask.corer dask.highlevelgraphr dask.utilsrrr/r>r2r3rirrrrsd"""""" 33333333,,,,,,......%%%%%%%% H;H;H;V>.J4J4J4J4J4J4r