Ug֌dZddlmZddlmZddlmZmZddlZ ddl m Z ddl m Z mZ ddlmZdd lmZ ddlZd Zd Zd ZdZdZdZdZdZdZdZe j@jCe j@j!dggdggZ"dZ#e$eeZ%e$eeeeeeeeZ&dZ'd dZ( d-dZ) d.dZ*eree*ndZ+eddfdZ,edddfdZ-dZ.d Z/d!Z0d"Z1e d#Z2d$Z3ed%Z4ed&Z5ee5ee4iZ6ed'Z7ed(Z8ed)Z9ed*Z:ed+Z;ee:ee7ee7ee8ee8ee9ee;ee;iZ "q1~ u6\5LE5vuV|$D ;T 5E!%ZtFF ++   &/ &/  GQFO ' #0 Nch| |jn|}||S|j\}}t||z }t||z } |\} } d} |jj} | |z| | zz| z|kDr"| r| dz} n| dz} | } | |z| | zz| z|kDr"| dk(s| dk(r|| z| z}t d|z| | fS)a! Attempts to compute a chunksize for the resampling output array that is as close as possible to the input array chunksize, while also respecting the maximum memory constraint to avoid loading to much data into memory at the same time. Parameters ---------- src : dask.array.Array The source array to resample w : int New grid width h : int New grid height chunksize : tuple(int, int) (optional) Size of the output chunks. By default the chunk size is inherited from the *src* array. max_mem : int (optional) The maximum number of bytes that should be loaded into memory during the regridding operation. Returns ------- chunksize : tuple(int, int) Size of the output chunks. Trra7Given the memory constraints the resampling operation could not find a chunksize that avoids loading too much data into memory. Either relax the memory constraint to a minimum of %d bytes or resample to a larger grid size. Note: A future implementation could handle this condition by declaring temporary arrays.) chunksizeshapefloatdtypeitemsize ValueError)srcr%r&rRmax_memstart_chunksizeshswheight_fractionwidth_fractionchcwdimnbytesmin_mems rNcompute_chunksizerds6(1'8cmmiO YYFBBikO2Yq[N FB C YY  F  R.%8 9F Bg M  !GB !GBg  R.%8 9F Bg M  Qw"'!N2V; - 07  78 8 r6MrPc t tdt|||||} | |j}t |j ||f| } i} | j D]n\\} } }|d}|d\}}|d\}}|d}|||||f}t||d|d|||||d|d  }||d|df|j|d|d | | | f<pt| j d }g}|D]]\} }tj|Dcgc]%\}}tj|d |d |d'c}}d}|j|_tj|d}| |j|k7r|j|}|Scc}}w)a A distributed version of 2-d grid resampling which operates on dask arrays and performs regridding on a chunked array. Parameters ---------- src : dask.array.Array The source array to resample w : int New grid width h : int New grid height ds_method : str (optional) Grid cell aggregation method for a possible downsampling (one of the *DS_* constants). us_method : str (optional) Grid cell interpolation method for a possible upsampling (one of the *US_* constants, optional). fill_value : scalar (optional) If None, numpy's default value is used. mode_rank : scalar (optional) The rank of the frequency determined by the *ds_method* ``DS_MODE``. One (the default) means most frequent value, two means second most frequent value, and so forth. chunksize : tuple(int, int) (optional) Size of the output chunks. By default this the chunk size is inherited from the *src* array. max_mem : int (optional) The maximum number of bytes that should be loaded into memory during the regridding operation. Returns ------- resampled : dask.array.Array A resampled version of the *src* array. z+dask is required for distributed regriddingr*r#r$r)r%r&r'r()arrayrSrUr*r)c|ddS)Nr)r#s rNz)resample_2d_distributed..s 1arPrfrSrUrr)da ImportErrorrdrRrOrSitems_resample_2d_delayedrUr concatenate from_delayedappendrechunk)rXr%r& ds_method us_method fill_value mode_rankrRrY temp_chunks chunk_mapr0r<rEchunkindsrHrIr?r@r) chunk_array resampledrowscolsrow_s rNresample_2d_distributedrsN zGHH#CAy'BKMM 399q!fk:IJ"* AT{#Y d#Y dEl$t)T$Y./ (SXs3xIItIYI #hC)YY+  Aq6+" :##%'8 9D D3nn!5 OOE'NE'NE'N K!"#% C  ..q !C)!;kk)$ J!s*F c $t| |||f} | |St|\} } t||| }t|}t|}t |t jjr |j}t|| | ||||||| } t| ||S)a Resample a 2-D grid to a new resolution. Parameters ---------- src : np.ndarray The source array to resample w : int New grid width h : int New grid height ds_method : str (optional) Grid cell aggregation method for a possible downsampling (one of the *DS_* constants). us_method : str (optional) Grid cell interpolation method for a possible upsampling (one of the *US_* constants, optional). fill_value : scalar (optional) If ``None``, it is taken from **src** if it is a masked array, otherwise from *out* if it is a masked array, otherwise numpy's default value is used. mode_rank : scalar (optional) The rank of the frequency determined by the *ds_method* ``DS_MODE``. One (the default) means most frequent value, zwo means second most frequent value, and so forth. x_offset : tuple(float, float) (optional) Offsets for the x-axis indices in the source array (useful for distributed regridding where chunks are not aligned with the underlying array). y_offset : tuple(float, float) (optional) Offsets for the x-axis indices in the source array (useful for distributed regridding where chunks are not aligned with the underlying array). out : numpy.ndarray (optional) Alternate output array in which to place the result. The default is *None*; if provided, it must have the same shape as the expected output. Returns ------- resampled : numpy.ndarray or dask.array.Array A resampled version of the *src* array. ) _get_out _get_mask_get_fill_valueupsample_methodsdownsample_methods isinstancenpma MaskedArraydata _resample_2d _mask_or_not) rXr%r&rrrsrtrux_offsety_offsetr)ruse_maskr{s rN resample_2dr!s\ 3aV $C { s^ND( S#6Jy)I +I#ruu(()hhS$)Y'HhMI  3 33rPc t||||f}||St|\}}t|||}|tvr t dt|}|||||dd|} t | ||S)a Upsample a 2-D grid to a higher resolution by interpolating original grid cells. src: 2-D *ndarray* w: *int* Grid width, which must be greater than or equal to *src.shape[-1]* h: *int* Grid height, which must be greater than or equal to *src.shape[-2]* method: one of the *US_* constants, optional Grid cell interpolation method fill_value: *scalar*, optional If ``None``, it is taken from **src** if it is a masked array, otherwise from *out* if it is a masked array, otherwise numpy's default value is used. out: 2-D *ndarray*, optional Alternate output array in which to place the result. The default is *None*; if provided, it must have the same shape as the expected output. Returns ------- upsampled : numpy.ndarray or dask.array.Array An upsampled version of the *src* array. invalid upsampling methodrr)rrrUPSAMPLING_METHODSrWr) rXr%r&methodrtr)rrupsampling_method upsampleds rN upsample_2drcs0 3aV $C { s^ND( S#6J ''455*62! T8Z>I  3 33rPc |tk(r|dkr tdt||||f}||St|\}}t |||}|t vr tdt |} | ||||||dd| } t | ||S)a Downsample a 2-D grid to a lower resolution by aggregating original grid cells. Parameters ---------- src : numpy.ndarray or dask.array.Array The source array to resample w : int New grid width h : int New grid height ds_method : str (optional) Grid cell aggregation method for a possible downsampling (one of the *DS_* constants). fill_value : scalar (optional) If ``None``, it is taken from **src** if it is a masked array, otherwise from *out* if it is a masked array, otherwise numpy's default value is used. mode_rank : scalar (optional) The rank of the frequency determined by the *ds_method* ``DS_MODE``. One (the default) means most frequent value, two means second most frequent value, and so forth. out : numpy.ndarray (optional) Alternate output array in which to place the result. The default is *None*; if provided, it must have the same shape as the expected output. Returns ------- downsampled : numpy.ndarray or dask.array.Array An downsampled version of the *src* array. rzmode_rank must be >= 1invalid downsampling methodr)DS_MODErWrrrDOWNSAMPLING_METHODSr) rXr%r&rrtrur)rrdownsampling_method downsampleds rN downsample_2drsBY]122 3aV $C { s^ND( S#6J ))677.v6% T8VZFK  S* 55rPc|!tj||jS|j|k7r t d|j|jk(ry|S)NrUz"'shape' and 'out' are incompatible)rzerosrUrSrW)r)rXrSs rNrrsL {xxSYY// 99 AB B 99 ! rPct|tjjr?tjj |}|tjj ur|dfSt dfS)NTF)rrrrgetmasknomask _NOMASK2D)rXrs rNrrsM#ruu(()uu}}S! ruu|| #:  e rPcnt|tjjrt|tjjsltj|r#tjj ||d}n!tjj |d}|j||S|S)NF)copy)rrrrisfinite masked_equalmasked_invalidset_fill_value)r)rXrtmaskeds rNrrs#ruu(()#ruu001{{:&++C%+H--c->  ! !* -M JrPcB|t|tjjr|j}|St|tjjr|j}|Stjj dgdg|j j}|S)NrF)rrU)rrrrrtrfrU)rtrXr)s rNrrs c255,, -J  RUU.. /J aSwciiHSSJ rPc|jd}|jd}|jd}|jd}||||fS)N)rS)rXr)src_wsrc_hout_wout_hs rN_get_dimensionsrsE IIbME IIbME IIbME IIbME % %%rPc  t|| \} } } } |\}}|\}}| |z |z }| |z |z }|tvr td|tvr tdt|}t|}| dk(s| dk(s | dk(s| dk(r#t j | | f|j S| |kr| |kr|||||||||| S| |krV| |kDrAt j | | f|j }|||||||||| }|||||||| S|||||||||| S| |krV| |kDrAt j | | f|j }|||||||||| }|||||||| S|||||||||| S| |kDs| |kDr|||||||| S|S)Nrrrr)rrrWrrrrU)rXrrrrrsrtrurrr)rrrrrLrMrCrDsrc_wosrc_horrtemps rNrrs#!0c!:E5%NFFNFFfnv%Ffnv%F**455 . .677.y9*95 zUaZ5A:!hhu~SYY77 EFN"3h #-y(#+S2 2  6>88UEN#)) >'sD(I'19h'/6 6  6>88UEN#)) >'sD(I'19h'/6 6 56> dHj!)8c; ; JrPct||\}}} } |\} } |\} }|| z | z }|| z |z }|| k(r|| k(r|S| |ks| |kr td|| z }|| z }t| D]h}t||z| z}t | D]G}t||z| z}|||f}t j |r|r|||fs||||f<A||||f<Ij|S)Ninvalid target sizerrWrintr,rr)rXrrrtrrr)rrrrrLrMrCrDscale_xscale_yout_ysrc_yout_xsrc_xvalues rN_upsample_2d_nearestr's!0c!:E5%NFFNFF V^f $E V^f $E ~%5.  u} .//emGemGWu_./5\E5F23Eu %E{{5!8UE\8J$)E5L!$.E5L! " JrPc\t||\}}} } |\} } |\} }|| z | z }|| z |z }|| k(r|| k(r|S| |ks| |kr td|dz | dkDr| dz ndz }|dz | dkDr| dz ndz }t| D]}||z| z}t|}||z }|dz}||k\r|}t | D]v}||z| z}t|}||z }|dz}||k\r|}|||f}|||f}|||f}|||f} |r}t j |xr|||f }!t j |xr|||f }"t j |xr|||f }#t j | xr|||f }$nTt j |}!t j |}"t j |}#t j | }$|!r*|"r(|#r&|$r$d}%||||z zz}&||| |z zz}'|&||'|&z zz}(n"|dkr|dkr|!}%|}(n|#}%|}(n|dkr|"}%|}(n|$}%| }(|%r |(|||f<p||||f<y|S)Nr?rT?r))rXrrrtrrr)rrrrrLrMrCrDrrrrrsrc_yfsrc_y0wysrc_y1rsrc_xfsrc_x0wxsrc_x1v00v01v10v11v00_okv01_okv10_okv11_okokv0v1rs) rN_upsample_2d_linearrDs!0c!:E5%NFFNFFfnv%Ffnv%F 6U?  u} .//|DG|DGE/V+V f_! U?F5\Eo/F[F&BaZFffn%Cffn%Cffn%Cffn%CS)F$vv~2F.FS)F$vv~2F.FS)F$vv~2F.FS)F$vv~2F.FS)S)S)S)&V2s++2s++R27^+c8BEBE8BEBE$)E5L!$.E5L!["j JrPc t||\} } } } | | k(r| | k(r|S| | kDs| | kDr td|\} }|\}}| | z |z | z }| |z |z | z }t| D]}||z|z}||z}t|}t|}||z }|tkr ||kDr|dz}t | D]}||z| z}||z}t|}t|}||z }|tkr ||kDr|dz}d}|} t ||dzD]P}!t ||dzD]8}"||!|"f}#t j|#s |r||!|"fr*|#} |tk(s6d}n|sPn| |||f<|S)NrrFT) rrWrr_EPSr,rrDS_FIRST)$rXrrrrtrurrr)rrrrrLrMrCrDrrrsrc_yf0src_yf1rrwy1rsrc_xf0src_xf1rrwx1donerrrvs$ rN_downsample_2d_first_lastrs"1c!:E5% ~%5.  u} .//NFFNFFv~&%/Gv~&%/GU?f,G#WW :&6/ aKF5\E&0G'G\F\FF"CTzfvo! DEvvz2"66A:6EE5L)A{{1~xD>> infer_interval_breaks(np.arange(5)) array([-0.5, 0.5, 1.5, 2.5, 3.5, 4.5]) >>> infer_interval_breaks([[0, 1], [3, 4]], axis=1) array([[-0.5, 0.5, 1.5], [ 2.5, 3.5, 4.5]]) rrr)axisrc3VK|] }|k(r tddn td"yw)Nr)slice).0nr s rN z(infer_interval_breaks..s.20!*+deD"od C0s&))cupyrndarrayrasarrayr-rfrUdifftaketupler,ndimrn)coordr deltasrr trim_lasts ` rNinfer_interval_breaksrs 5$,,/  5! 5zQxx%++.. 2775t, ,F GGEA3T *RWWVaSt-L LE 7752$T *RWWVbT-M MD2$UZZ022I >>5% "2V";TB NNrP)rrNrNN)rrNrrrN)r)>__doc__ __future__r itertoolsrmathrrnumpyrnumbarutilsr r dask.arrayrfrj dask.delayedr rkr US_NEAREST US_LINEARrDS_LASTrDS_MAXDS_MEANrDS_VARrr getmaskarrayrrdictrrrOrdrrrmrrrrrrrrrrrrrrrr rrrhrPrNr-s4#($             EE  ruu{{QC5u{= >  *Y?wW&fF$f6 6r4nDLDH$(L^8@7=%)<4~07w{+D#,$$4N%,t06f  &&3l8DDN !4 "68,,^11hRRj55p::z !4 ";!: 6 6!4 6 68OIKB  Ds" EE EEEE