a ;ghqe@sddlmZddlmZddlZddlmZddlZddlZddl Z ddl Z ddl Z ddl Z ddlm ZddlZddlmZmZmZmZddlmZddlmZdd lmZddlm Zdd lmZmZd a e !d Z"d dZ#d=ddZ$d>ddZ%ddZ&eddZ'eddZ(ddZ)ddZ*ddZ+d d!Z,d"d#Z-d$d%Z.d&d'Z/e-dd dd(fd)d*Z0d?d+d,Z1d d-dd.d/d0Z2d@d2d3Z3d4d5Z4d6d7Z5d8d9Z6e7d:d;d<Z8dS)A) OrderedDict)contextmanagerN) zip_longest)tqdm) device_putjitlaxvmap)Tracer) host_callback) ravel_pytree) tree_flattentree_mapFz\d+$cCst|tj|dS)z Initializes internal state for the Python and NumPy random number generators. :param int rng_seed: seed for Python and NumPy random states. N)randomseednp)Zrng_seedr\/mounts/lovelace/software/anaconda3/envs/metaDMG/lib/python3.9/site-packages/numpyro/util.py set_rng_seeds rTcCs"|stdd}tjd|dS)z Changes the default array type to use 64 bit precision as in NumPy. :param bool use_x64: when `True`, JAX arrays will use 64 bits by default; else 32 bits. ZJAX_ENABLE_X64rZjax_enable_x64Nosgetenvjaxconfigupdate)Zuse_x64rrr enable_x64's rcCs&|durtdd}tjd|dS)z Changes platform to CPU, GPU, or TPU. This utility only takes effect at the beginning of your program. :param str platform: either 'cpu', 'gpu', or 'tpu'. NZJAX_PLATFORM_NAMEcpuZjax_platform_namer)platformrrr set_platform3s rcCs>tdd}tdd|}dd|g|tjd<dS)a By default, XLA considers all CPU cores as one device. This utility tells XLA that there are `n` host (CPU) devices available to use. As a consequence, this allows parallel mapping in JAX :func:`jax.pmap` to work in CPU platform. .. note:: This utility only takes effect at the beginning of your program. Under the hood, this sets the environment variable `XLA_FLAGS=--xla_force_host_platform_device_count=[num_devices]`, where `[num_device]` is the desired number of CPU devices `n`. .. warning:: Our understanding of the side effects of using the `xla_force_host_platform_device_count` flag in XLA is incomplete. If you observe some strange phenomenon when using this utility, please let us know through our issue or forum page. More information is available in this `JAX issue `_. :param int n: number of CPU devices to use. Z XLA_FLAGSz*--xla_force_host_platform_device_count=\S+ z)--xla_force_host_platform_device_count={}N)rrresubsplitjoinformatenviron)nZ xla_flagsrrrset_host_device_count?s r(ccs:|r0|dVWdq61s$0YndVdS)zJ Optionally wrap inside `context_manager` if condition is `True`. Nr) conditionZcontext_managerrrroptional[s&r*ccs"t}zdadVW|an|a0dS)NT)_DISABLE_CONTROL_FLOW_PRIM)Z stored_flagrrrcontrol_flow_prims_disabledgs r,cCs0tr|r||S||Snt|||||SdSN)r+rcond)predZ true_operandZtrue_funZ false_operandZ false_funrrrr.rs  r.cCs0tr|}||r||}q|St|||SdSr-)r+r while_loop)Zcond_funbody_funinit_valvalrrrr0|s  r0cCsBtr.|}tt|t|D]}|||}q|St||||SdSr-)r+rangeintr fori_loop)lowerupperr1r2r3irrrr6s  r6cCs t|t S)zf Checks if `x` is not an array generated inside `jit`, `pmap`, `vmap`, or `lax_control_flow`. ) isinstancer xrrrnot_jax_tracersr=cOs|Sr-r)r<argskwargsrrridentitysr@cs(dtdt_fdd}|S)N_cachecsJj}|vr&|}|=||<n||<t|krF|jdd|S)NF)last)rBlenpopitem)fnZfn_cachekeysmax_sizeouter_fnrr_wrappeds   zcached_by.._wrapped)getattrrrB)rJrHrKrrGr cached_bys rMcsdkrtdndigtD]*}tt|d|<|jdddq2fddfd d fd d fd d}|S)zvFactory that builds a progress bar decorator along with the `set_tqdm_description` and `close_tqdm` functions )positionz Compiling.. TrefreshcsLtt|}|sJt|}|jd|dd||dS)NzRunning chain FrQ) _CHAIN_REsearchstrr5groupset_descriptionrarg transformZdeviceZ chain_matchchain) tqdm_barsrr _update_tqdms  z*progress_bar_factory.._update_tqdmcsdtt|}|sJt|}|||tkr`tD]}| qNdSr-) rSrTrUr5rVrappendrDr4closerX)finished_chains num_chainsr\rr _close_tqdms    z)progress_bar_factory.._close_tqdmcstjdkfddfdddd}tjdkfddfd ddd}tjkfd dfd ddd}dS) zUpdates tqdm progress bar of a JAX loop only if the iteration number is a multiple of the print_rate Usage: carry = progress_bar((iter_num, print_rate), carry) rOcstjdddS)NrTresultZtap_with_devicer Zid_tap_)r]iter_numrrszDprogress_bar_factory.._update_progress_bar..csSr-rrfrhrrriN)operandrcstjddSNTrcrerf)r]rh print_raterrriscsSr-rrfrjrrrirkcstjddSrmrerf)rbrh remainderrrriscsSr-rrfrjrrrirk)rr.)rhrg)rbr] num_samplesrnrorjr_update_progress_bars$     z2progress_bar_factory.._update_progress_barcsfdd}|S)aDecorator that adds a progress bar to `body_fun` used in `lax.fori_loop`. Note that `body_fun` must be looping over a tuple who's first element is `np.arange(num_samples)`. This means that `iter_num` is the current iteration number cs||}|d|SNrOr)r9valsrd)rqfuncrrwrapper_progress_bars  zRprogress_bar_factory..progress_bar_fori_loop..wrapper_progress_barr)rtru)rq)rtrprogress_bar_fori_loopsz4progress_bar_factory..progress_bar_fori_loop)r5r4 tqdm_autorW)rprar[rvr) rbrqr]r`rarprnror\rprogress_bar_factorys    rxrOc  s||ks J|dksJ|dur,|||n|}||||ksDJt|\} } ||||} | dd} | dkrtdkrtjdtdd}ttfdd }t j |f| j | j d }|st d ||||| |f\}}}}n| dkr,t|| }||}t d ||||| |f\}}}}n| d d}| d dd}||t| t|f}|d krvt|d |ntt|X}|D]B}t|||}|j||dd|r|j||d ddqWdn1s0Y|\}}}}t| |}|r||fS|S)a This looping construct works like :func:`~jax.lax.fori_loop` but with the additional effect of collecting values from the loop body. In addition, this allows for post-processing of these samples via `transform`, and progress bar updates. Note that, `progbar=False` will be faster, especially when collecting a lot of samples. Refer to example usage in :func:`~numpyro.infer.mcmc.hmc`. :param int lower: the index to start the collective work. In other words, we will skip collecting the first `lower` values. :param int upper: number of times to run the loop body. :param body_fun: a callable that takes a collection of `np.ndarray` and returns a collection with the same shape and `dtype`. :param init_val: initial value to pass as argument to `body_fun`. Can be any Python collection type containing `np.ndarray` objects. :param transform: a callable to post-process the values returned by `body_fn`. :param progbar: whether to post progress bar updates. :param bool return_last_val: If `True`, the last value is also returned. This has the same type as `init_val`. :param thinning: Positive integer that controls the thinning ratio for retained values. Defaults to 1, i.e. no thinning. :param int collection_size: Size of the returned collection. If not specified, the size will be ``(upper - lower) // thinning``. If the size is larger than ``(upper - lower) // thinning``, only the top ``(upper - lower) // thinning`` entries will be non-zero. :param `**progbar_opts`: optional additional progress bar arguments. A `diagnostics_fn` can be supplied which when passed the current value from `body_fun` returns a string that is used to update the progress bar postfix. Also a `progbar_desc` keyword argument can be supplied which is used to label the progress bar. :return: collection with the same type as `init_val` with values collected along the leading axis of `np.ndarray` objects. rONraZgpuzRWe will disable progress bar because it does not work yet on multi-GPUs platforms. stacklevelFcsL|\}}}|||tdk|fdd|t}|||fS)Nrcs|jtdS)Nr)atsetr r;)idxrZr3rrriHrkz0fori_collect.._body_fn..)r.r@)r9rs collection start_idxthinningr1rZ)r}r3r_body_fn@s  zfori_collect.._body_fn)dtyperdiagnostics_fn progbar_desccSsdS)Nrrr;rrrri]rkzfori_collect..rQ)r poprZdefault_backendwarningswarnfind_stack_levelrM fori_collectjnpZzerosshaperr6rxrrrZtrangerWZset_postfix_strr )r7r8r1r2rZZprogbarZreturn_last_valZcollection_sizerZ progbar_optsZ init_val_flatZ unravel_fnrrarr~Zlast_valrgrvZ _body_fn_pbarrrrstr9Zunravel_collectionrrrrsV-          :  rcs`t|d}t|dd|ddD]}t|dks.Jq.tt}dkrnfndtfdd|}durntdkrtfdd|}tdk}|dkrdndtfd d|}t|}|dkrt ||n||}t|dktdktfd d|}tfd d|S) a> Vectorizing map that maps a function `fn` over `batch_ndims` leading axes of `xs`. This uses jax.vmap over smaller chunks of the batch dimensions to keep memory usage constant. :param callable fn: The function to map over. :param xs: JAX pytree (e.g. an array, a list/tuple/dict of arrays,...) :param int batch_ndims: The number of leading dimensions of `xs` to apply `fn` element-wise over them. :param int chunk_size: Size of each chunk of `xs`. Defaults to the size of batch dimensions. :returns: output of `fn(xs)`. rNrOrcst|t|dSr-rreshaperr;) batch_ndims prepend_shaperrrirkzsoft_vmap..cs$t|dffdt|dS)Nr))rrrO)rpadrndimr;)rrrrirk)cs$t|ft|ddSrrrr;) chunk_sizerrrrirkc s@t|ttt|dft|ddSr-)rrr5rprodry) batch_size map_ndimsrrris0cst|t|ddSrrrr) batch_shaperrrirk) r rrr5rrminr rmap)rFZxsrrZ flatten_xsr<Z num_chunksZysr)rrrrrrrr soft_vmapqs:      rz Trace Shapes:)compute_log_probtitle last_sitec Cs,|s |S|gg}|dg|D]H\}}|ddkrd||dgddt|ddd D||kr(qrq(|d g|D]\}}|dd krt|d d d }t|d dd }||ddgdd|Dddgdd|Dt|} t|ddd } | dt| | }| t| | d}|ddgdd|Dddgdd|Dt|sx|st|r||rt|d |ddd }|ddgdd|DddgnH|ddkrt|ddd } ||ddgdd| Dddg||krq$qt|S)a Given the trace of a function, returns a string showing a table of the shapes of all sites in the trace. Use :class:`~numpyro.handlers.trace` handler (or funsor :class:`~numpyro.contrib.funsor.enum_messenger.trace` handler for enumeration) to produce the trace. :param dict trace: The model trace to format. :param compute_log_prob: Compute log probabilities and display the shapes in the table. Accepts True / False or a function which when given a dictionary containing site-level metadata returns whether the log probability should be calculated and included in the table. :param str title: Title for the table of shapes. :param str last_site: Name of a site in the model. If supplied, subsequent sites are not displayed in the table. Usage:: def model(*args, **kwargs): ... with numpyro.handlers.seed(rng_seed=1): trace = numpyro.handlers.trace(model).get_trace(*args, **kwargs) print(numpyro.util.format_shapes(trace)) z Param Sites:typeparamNcSsg|] }t|qSrrU.0sizerrr rkz!format_shapes..valuerrz Sample Sites:samplerFr event_shapez distcSsg|] }t|qSrrrrrrrrk|cSsg|] }t|qSrrrrrrrrkcSsg|] }t|qSrrrrrrrrkcSsg|] }t|qSrrrrrrrrklog_probcSsg|] }t|qSrrrrrrrrkplatez platecSsg|] }t|qSrrrrrrrrk)rHr^itemsrLrDcallabler _format_table) tracerrrrowsnamesiterr event_dimrrrr format_shapessz!         "rloosec Cs |dvs J|D]\}}|ddkrt|d}tt|djt|dd|t|dj }t dd|dD}t|}t |D]}| d } || d kr| |vr| | kr| d d rqd | d |d} |dkrt | q|dkst|dkrtj| tdqqdS)N)rstricterrorrrrrFcss|] }|jVqdSr-)dim)rfrrr rkz"_validate_model..Zcond_indep_stackrOZ_control_flow_doneFz.Missing a plate statement for batch dimension z at site 'z_'. You can use `numpyro.util.format_shapes` utility to check shapes at all sites of your model.rrrry)rrrrZbroadcast_shapestuplerrrDrr|r4get ValueErrorrrr) model_traceZ plate_warningrrZ value_ndimrZ plate_dimsZ batch_ndimr9rmessagerrr_validate_models.   "   rc Cstdd|D}tdd|D}tdd|D}t}||@rjtjd||@td|||Bkstjd|||td|||Bkstjd|||td||@D]}||}||}t|d d r.t|d d r.|d j|d jkr.td ||d j|d jt|d d rt|d d r|d  |d d} |d  |d d} | | krqt t | t | ddD]&\} } | | krtd|| | qqtdd|D}tdd|D}||kstjd||tddS)a Checks the following assumptions: 1. Each sample site in the model also appears in the guide and is not marked auxiliary. 2. Each sample site in the guide either appears in the model or is marked, auxiliary via ``infer={'is_auxiliary': True}``. 3. Each :class:`~numpyro.primitives.plate` statement in the guide also appears in the model. 4. At each sample site that appears in both the model and guide, the model and guide agree on sample shape. :param dict model_trace: The model trace to check. :param dict guide_trace: The guide trace to check. :raises: RuntimeWarning, ValueError css.|]&\}}|ddkr|dds|VqdSrrZ is_observedFNrrrrrrrr5sz*check_model_guide_match..css0|](\}}|ddkr|ddr|VqdS)rrZinferZ is_auxiliaryNrrrrrr:s css.|]&\}}|ddkr|dds|VqdSrrrrrrr@sz%Found auxiliary vars in the model: {}ryzjFound non-auxiliary vars in guide but not model, consider marking these infer={{'is_auxiliary': True}}: {}z%Found vars in model but not guide: {}rFrz:Model and guide event_dims disagree at site '{}': {} vs {}rr?Z sample_shaperO) fillvaluez6Model and guide shapes disagree at site '{}': {} vs {}css"|]\}}|ddkr|VqdSrrNrrrrrr}scss"|]\}}|ddkr|VqdSrrrrrrrsz1Found plate statements in guide but not model: {}N) r|rrrr%rhasattrrrrrreversed) rZ guide_traceZ guide_varsZaux_varsZ model_varsZ enum_varsrZ model_siteZ guide_siteZ model_shapeZ guide_shapeZ model_sizeZ guide_sizerrrcheck_model_guide_match#s          rcs8gd}|D]`}gd}d}|D]&}|dur6|d7}q ||d7<q tdD]}t||||||<qPq t|D]d\}}gggg}d}|D]$}|dur|d7}q|||qddt||dD}t|g||<qvdgt|d|D],}t|D]\}}t|t||<qqd fd d |DS) zI Formats a right justified table using None as column separator. )rrrrNrOcSsFg|]>\}}}|dkr,dg|t||n|dg|t|qS)rr)rD)rwidthcol directionrrrrsz!_format_table..Zrrl c3s(|] }dddt|DVqdS)r css|]\}}||VqdSr-)rjust)rcellrrrrrrkz*_format_table...N)r$zip)rrowZ cell_widthsrrrsz _format_table..)r4max enumerater^rsumrDr$)rZ column_widthsrwidthsjrr9colsrrrrs8     rcCstdd|dDS)z :param str version: Version, in string format. Parse version string into tuple of ints. Only to be used for the standard 'major.minor.patch' format, such as ``'0.2.13'``. Source: https://stackoverflow.com/a/11887825/4451315 cSsg|] }t|qSr)r5)rnumberrrrrrkz!_versiontuple...)rr#)versionrrr _versiontuples r)returncCsRddl}tj|j}t}d}|rNt|}||rN|j }|d7}q"qNq"|S)z Find the first place in the stack that is not inside numpyro (tests notwithstanding). Source: https://github.com/pandas-dev/pandas/blob/ccb25ab1d24c4fb9691270706a59c8d319750870/pandas/util/_exceptions.py#L27-L48 rNrO) numpyrorpathdirname__file__inspect currentframegetfile startswithf_back)rZpkg_dirframer'fnamerrrrs   r)T)N)rON)r)9 collectionsr contextlibrr itertoolsrrrr!rnumpyrrZ tqdm.autorwrrrrr Zjax.corer Zjax.experimentalr Zjax.flatten_utilr Z jax.numpyrZ jax.tree_utilr rr+compilerSrrrr(r*r,r.r0r6r=r@rMrxrrrrrrrr5rrrrrsd               V p 7 _ i/